Bug: Docker containers failing (libgomp.so.1)

[lunfardo]

What happened?

Trying to run to llama.cpp in a container but so far it's been unsuccessful.

> docker run -v  /ssdpool/llm-models:/models  ghcr.io/ggerganov/llama.cpp:light -m /models/Meta-Llama-3-8B.Q8_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512

gets me

/main: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory

I have CUDA installed:

> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

I have also installed nvidia-container-toolkit and followed these instructions

I get the same error when I create the image myself from master (local/llama.cpp:light-cuda)

Name and Version

Docker

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

[lunfardo]

ghcr.io/ggerganov/llama.cpp:full-cuda--b1-bde7cd3 works though

 docker run --runtime=nvidia --gpus all -v /ssdpool/llm-models:/models ghcr.io/ggerganov/llama.cpp:full-cuda--b1-bde7cd3 --run -m /models/Meta-Llama-3-8B.Q8_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 -
ngl 9999
Log start
main: build = 0 (unknown)
main: built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
main: seed  = 1717494894
llama_model_loader: loaded meta data with 22 key-value pairs and 291 tensors from /models/Meta-Llama-3-8B.Q8_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = .
llama_model_loader: - kv   2:                           llama.vocab_size u32              = 128256
llama_model_loader: - kv   3:                       llama.context_length u32              = 8192
llama_model_loader: - kv   4:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   5:                          llama.block_count u32              = 32
llama_model_loader: - kv   6:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   7:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   8:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   9:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv  10:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  11:                       llama.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv  12:                          general.file_type u32              = 7
llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  14:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  15:                      tokenizer.ggml.scores arr[f32,128256]  = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
llama_model_loader: - kv  18:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 128001
llama_model_loader: - kv  20:                    tokenizer.chat_template str              = {% set loop_messages = messages %}{% ...
llama_model_loader: - kv  21:               general.quantization_version u32              = 2

[sides-flow]

I also received a similar error when trying to build release 3077 (3076 works):

 => [build-llama 5/5] RUN cd llama.cpp && mkdir build && cd build && cmake ../ -DLLAMA_STATIC=ON && cmake --build . --config Release -j                                                                      32.8s
 => => # [  8%] Linking CXX executable ../../bin/gguf
 => => # /usr/bin/ld: attempted static link of dynamic object `/usr/lib/gcc/x86_64-linux-gnu/11/libgomp.so'
 => => # collect2: error: ld returned 1 exit status
 => => # gmake[2]: *** [examples/gguf/CMakeFiles/gguf.dir/build.make:109: bin/gguf] Error 1
 => => # gmake[1]: *** [CMakeFiles/Makefile2:3093: examples/gguf/CMakeFiles/gguf.dir/all] Error 2
 => => # gmake[1]: *** Waiting for unfinished jobs....

[goldyfruit]

Same here.

/server: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory

In my case I just edited https://github.com/ggerganov/llama.cpp/blob/master/.devops/server-cuda.Dockerfile#L33 and added libgomp1.

[slaren]

A PR to add the missing library would be welcome. Alternatively, you can build with LLAMA_NO_OPENMP, but performance may be worse.

[kschen202115]

add this 'apt-get install -y libcurl4-openssl-dev libgomp1' on line 22 like this https://github.com/kschen202115/build_llama.cpp_docker/blob/main/Dockerfile

[huynhbaobk]

Same here.

/server: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory

In my case I just edited https://github.com/ggerganov/llama.cpp/blob/master/.devops/server-cuda.Dockerfile#L33 and added libgomp1.

Thanks. It works for me.

RUN apt-get update &&
apt-get install -y libcurl4-openssl-dev libgomp1

[0x4139]

The last working version is ghcr.io/ggerganov/llama.cpp:server-cuda-0b832d53ba0ffcc759c8d62ede3772dd62321f8e before a fix comes by.

Just in case someone needs it.