Open
Description
Hello, I'm trying to run this on Windows 11, and after fixing the cmake errors with #106 by adding the definitions, when I try to load the 30B model I get a lot of title errors followed by a Segmentation fault:
$ ./Release/chat.exe -m ggml-model-q4_0.bin
main: seed = 1679560202
llama_model_load: loading model from 'ggml-model-q4_0.bin' - please wait ...
llama_model_load: ggml ctx size = 1055.50 MB
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1120528208, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1120528208, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1120528208, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1129074932, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1129074932, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1175959936, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1175959936, available 1106773248)
[...many many lines more...]
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1131858388, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1131858388, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248)
ggml_new_tensor_impl: not enough space iSegmentation fault
I have 64Gb of memory and 53Gb are free when I try, so I'm guessing it should be enough? I see the same issue popping for someone in llama ggml-org#153 so it might be related.
I have also tested with 13B model and it won't load either:
$ ./Release/chat.exe -m ggml-alpaca-13b-q4.bin
main: seed = 1679561627
llama_model_load: loading model from 'ggml-alpaca-13b-q4.bin' - please wait ...
llama_model_load: ggml ctx size = 2767.49 MB
Segmentation fault
So I guess it's not a matter of me lacking memory:
Metadata
Metadata
Assignees
Labels
No labels