Skip to content

Bug: Can't quantize in gguf q5_k_m a mamba architecture codestral #8690

Closed
@Volko61

Description

@Volko61

What happened?

Error: Error converting to fp16: b'INFO:hf-to-gguf:Loading model: mamba-codestral-7B-v0.1\nTraceback (most recent call last):\n File "/home/user/app/llama.cpp/convert_hf_to_gguf.py", line 3673, in \n main()\n File "/home/user/app/llama.cpp/convert_hf_to_gguf.py", line 3645, in main\n model_architecture = hparams["architectures"][0]\nKeyError: 'architectures'\n'

image

Name and Version

latest (hf space)

What operating system are you seeing the problem on?

Other? (Please let us know in description)

Relevant log output

Error: Error converting to fp16: b'INFO:hf-to-gguf:Loading model: mamba-codestral-7B-v0.1\nTraceback (most recent call last):\n File "/home/user/app/llama.cpp/convert_hf_to_gguf.py", line 3673, in \n main()\n File "/home/user/app/llama.cpp/convert_hf_to_gguf.py", line 3645, in main\n model_architecture = hparams["architectures"][0]\nKeyError: 'architectures'\n'

![image](https://github.com/user-attachments/assets/6f1ab039-754d-4c6c-8bb8-274c0c99ca13)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug-unconfirmedlow severityUsed to report low severity bugs in llama.cpp (e.g. cosmetic issues, non critical UI glitches)stale

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions