Is official mixtral 8x22b working properly?

[mirek190]

Is official mixtral 8x22b working properly with llamacpp?

[phymbert]

Have you faced any issue ? it looks this is the same architecture, so I do not get your question

• convert.py : add consolidated.safetensors for mixtral 8x22b #6587
• Request Support for Mistral-8x22B #6580 (comment)

Looks like a dupplicate, reopen if I miss something

[rankaiyx]

Have you faced any issue ? it looks this is the same architecture, so I do not get your question

• convert.py : add consolidated.safetensors for mixtral 8x22b #6587
• Request Support for Mistral-8x22B #6580 (comment)

Looks like a dupplicate, reopen if I miss something

Its math ability seems to be poor. Is it because it is the basic model?

example:
./main -m ../gguf/Mixtral-8x22B-v0.1.IQ3_XS/Mixtral-8x22B-v0.1.IQ3_XS-00001-of-00005.gguf -n 500 -t 32 -p "Question:The department store entrusts the handling company to transport 1000 glass vases, and the freight for each glass vase is 1.50 yuan. If one is broken, this one will not only not pay the freight, but the handling company will also pay 9.50 yuan. The department store finally paid 1456 yuan. How many vases were broken during the handling? Answer:Let's verify step by step"

[ghogan42]

Its math ability seems to be poor. Is it because it is the basic model?

example:
./main -m ../gguf/Mixtral-8x22B-v0.1.IQ3_XS/Mixtral-8x22B-v0.1.IQ3_XS-00001-of-00005.gguf -n 500 -t 32 -p "Question:The department store entrusts the handling company to transport 1000 glass vases, and the freight for each glass vase is 1.50 yuan. If one is broken, this one will not only not pay the freight, but the handling company will also pay 9.50 yuan. The department store finally paid 1456 yuan. How many vases were broken during the handling? Answer:Let's verify step by step"

Mixtral-8x22B is a base model that has not been instruction tuned. Base models can only really do text completion and are bad at most everything else.

You'll have to wait until an official instruct tuned model is released or for 3rd parties to do additional fine-tuning.