Issue in convert-lora-to-ggml.py

[ragesh2000]

I was trying to create a ggml using lora weight using the following code.
python convert-lora-to-ggml.py --model /home/ragesh/Documents/conversion/mistral-7b-instruct-v0.1.Q8_0.gguf --lora /home/ragesh/Documents/conversion/adapter_model.bin
But it producing the following file not found error

Traceback (most recent call last):
  File "convert-lora-to-ggml.py", line 63, in <module>
    model = torch.load(input_model, map_location="cpu")
  File "/home/ragesh/miniconda3/envs/llamacpp/lib/python3.8/site-packages/torch/serialization.py", line 988, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/ragesh/miniconda3/envs/llamacpp/lib/python3.8/site-packages/torch/serialization.py", line 437, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/ragesh/miniconda3/envs/llamacpp/lib/python3.8/site-packages/torch/serialization.py", line 417, in __init__
    super().__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '--model/adapter_model.bin'

I am sure that the base model and lora weight are present at the specified path. Whats happening here ? Am I doing something wrong ?

[amygbAI]

Hi @ragesh2000 , this might help you out ..have explained a few things before closing the bug myself
#4896

[ragesh2000]

Hi @amygbAI To follow the blog you mentioned I should have the adapter_model.bin". But what I have is adapter_model.safetensors". How to convert this to .bin ?

[amygbAI]

Point #7 in the STEPS I have mentioned, while closing the bug

…

On Wed, Jan 24, 2024, 10:50 AM Ragesh K R ***@***.***> wrote: Hi @amygbAI <https://github.com/amygbAI> To follow the blog you mentioned I should have the adapter_model.bin". But what I have is adapter_model.safetensors". How to convert this to .bin ? — Reply to this email directly, view it on GitHub <#4940 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATIQOSGY7JELFVD3ROIYYLDYQCKYTAVCNFSM6AAAAABB2HNVOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBXGM4TMOBUGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ragesh2000]

what you mentioned was to save using torch.save(trainer.model.state_dict(), f"{script_args.output_dir}/adapter_model.bin"),
But the training is already done. Now how can i save trainer,model_state ? @amygbAI

[xuefeng-xu]

Merge the adapters into the base model: model = model.merge_and_unload()
See https://huggingface.co/docs/trl/main/en/use_model#use-adapters-peft

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.