Skip to content

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248) #130

Open
@Dannythm

Description

@Dannythm

Hello, I'm trying to run this on Windows 11, and after fixing the cmake errors with #106 by adding the definitions, when I try to load the 30B model I get a lot of title errors followed by a Segmentation fault:

$ ./Release/chat.exe -m ggml-model-q4_0.bin
main: seed = 1679560202
llama_model_load: loading model from 'ggml-model-q4_0.bin' - please wait ...
llama_model_load: ggml ctx size = 1055.50 MB
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1120528208, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1120528208, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1120528208, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1129074932, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1129074932, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1175959936, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1175959936, available 1106773248)
[...many many lines more...]
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1131858388, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1131858388, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 1178743392, available 1106773248)
ggml_new_tensor_impl: not enough space iSegmentation fault

I have 64Gb of memory and 53Gb are free when I try, so I'm guessing it should be enough? I see the same issue popping for someone in llama ggml-org#153 so it might be related.

I have also tested with 13B model and it won't load either:

$ ./Release/chat.exe -m ggml-alpaca-13b-q4.bin
main: seed = 1679561627
llama_model_load: loading model from 'ggml-alpaca-13b-q4.bin' - please wait ...
llama_model_load: ggml ctx size = 2767.49 MB
Segmentation fault

So I guess it's not a matter of me lacking memory:

image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions