Misc. bug: Shared libraries don't properly contain /common/ functions

[kinchahoy]

Name and Version

I'm using cppyy to create python bindings to the terrific, cleaned-up multimodal functionality in mtmd.cpp (great work @ngxson!)

I managed to get it working (happy to share some example code), but I had to create a custom Makefile to create a separate library for functions in /common because they don't seem to correctly get included in libllama.dylib (or .so).

Functions like "common_params_sampling" or "common_batch_add" are not available in libllama.dylib even after correctly building with BUILD_SHARED_LIBS=1.

Okay - after some digging, I think the correct thing to do is simply to have the common target respect the overall build settings. So instead of forcing static (like we do now)

add_library(${TARGET} STATIC

switch to

add_library(${TARGET}
// Equivalent to add_library(${TARGET} SHARED when BUILD_SHARED_LIBS=ON

This ensures that when you build with -DBUILD_SHARED_LIBS=ON, lib_common.so (or dylib) is created.

I don't believe there are performance concerns here (ran some quick benchmarks), but we could also consider a distinct flag to build lib_common.so.

I'll submit a PR to propose the simple version for now.

Operating systems

No response

Which llama.cpp modules do you know to be affected?

No response

Command line

Problem description & steps to reproduce

cmake -B build-DBUILD_SHARED_LIBS=ON

does not provide shared library access to common inferencing functions like common_batch_add (no libcommon.so created)

First Bad Commit

No response

Relevant log output

[ggerganov]

libcommon and libmtmd are not yet ready to be used by 3rd-party projects, so I am not sure this change is needed atm.

[ngxson]

Thanks for reporting. Yes the libmtmd.so should be independent from libcommon, this will be fixed when libmtmd has a pure C interface so 3rd party app can use it

[kinchahoy]

libcommon and libmtmd are not yet ready to be used by 3rd-party projects, so I am not sure this change is needed atm.

As a general approach, this makes sense, but it's worth noting that you pretty much cannot generate tokens using the shared libraries as currently built (I tried!), since they exclude simple and essential functions in libcommon (i.e. in common.cpp etc.). Those functions are also used in just about every code example in examples/.

If we allow folks to build shared libraries, it makes sense to also expose at least some of the core functions in libcommon.

Otherwise, folks using the shared libraries are forced to replicate those functions manually or modify to the make system. In fact, this is likely a major reason why llama-cpp-python is often unusably out of date.

Alternatively - perhaps we can temporarily add a new build flag to explicitly build libcommon for folks who need it?

[ngxson]

Ok I misunderstood your question. No I don't think you should use it that way, libcommon has breaking change very frequently and you will give yourself more headache if you use it

What I find working well is to simply copy whatever function in common.cpp that you use into your code, I done this in wllama which allow having a stable API for months now

[kinchahoy]

Ah gotcha - I was hoping to maintain less C but I guess that's going to be hard. For now I'll probably just build libcommon for myself. Hopefully we can create a simple layer for the basic overhead of token generation that can be a library soon (I'm happy to help if folks are interested in something like that, but I know @ggerganov is not a fan of such libraries being in the main repo) PS - I've been looking for a good webgpu / wasm solution for local inferencing. wllama is super interesting. Do you think it's within reach to support webgpu on it? Happy to try and help.

…

On Tue, Apr 29, 2025 at 11:36 AM Xuan-Son Nguyen ***@***.***> wrote: *ngxson* left a comment (ggml-org/llama.cpp#13156) <#13156 (comment)> Ok I misunderstood your question. No I don't think you should use it that way, libcommon has breaking change very frequently and you will give you more headache than not using it. What I find working well is to simply copy whatever function in common.cpp that you use into your code, I done this in wllama <https://github.com/ngxson/wllama/tree/master/cpp/helpers> which allow having a stable API for months now — Reply to this email directly, view it on GitHub <#13156 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRT7PJI3PM6Y7NOT6MCXZL237BDPAVCNFSM6AAAAAB4BNNVEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMZZHA3DKNZRHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>