Vocab size mismatch

[eswarthammana]

Llama_2_7B-chat
vocab size mismatch (model has -1 but tokenizer.model has 32000)

[yaashwardhan]

This is an easy fix.

There should be a .json file (probably params.json) inside the llama-2-7b-chat folder.
Open the json file and set the "vocab_size": to 32000 from -1.

Here is my params.json file:

{"dim": 4096, "multiple_of": 256, "n_heads": 32, "n_layers": 32, "norm_eps": 1e-06, "vocab_size": 32000}

[eswarthammana]

Thanks for the fix, i did downloaded older version repo where i have not faced this issue few days back. :)

[ishowshao]

same issue, why meta write -1 ?

[lorddoskias]

I also hit this and I think the correct way to fix it is in the convert script to simply remove the "vocab_size" if it equals -1 which will result in getting it from the tok_embeddings.weight.

[kaiwren]

+1 I ran into this also (Exception: Vocab size mismatch (model has -1, but ../llama/tokenizer.model has 32000)) today with llama-2-7b.

[guertsen]

Had the same issue with llama-2-7b.

[rsbepvb]

Confirming this fixed the issue with the most recent Llama download

[ursachec]

Ran into a similar issue today, using the current tip of master bcc0eb4:

$ python3 llama.cpp/convert.py ./Magicoder-S-DS-6.7B/ --outtype f16 --outfile magicoder-S-DS-6.7B.FP16.gguf
//...
Exception: Vocab size mismatch (model has 32256, but Magicoder-S-DS-6.7B/tokenizer.model combined with Magicoder-S-DS-6.7B/added_tokens.json has 32022)

[FotieMConstant]

I confirm this fixed the issue with Llama2 7b models