Skip to content

llama : move end-user examples to tools directory #13249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 2, 2025
Merged

Conversation

slaren
Copy link
Member

@slaren slaren commented May 1, 2025

Moves examples that are intended to be used by end users (rather than developers) to a separate tools directory. Eventually only these programs will be included in the binary distributions.

This is the tentative list of examples "promoted" to tools in this PR:

  • batched-bench
  • cvector-generator
  • export-lora
  • gguf-split
  • imatrix
  • llama-bench
  • llava
  • main
  • perplexity
  • quantize
  • rpc
  • run
  • server
  • tokenize
  • tts

@slaren slaren requested a review from ngxson as a code owner May 1, 2025 22:47
@slaren slaren requested a review from ggerganov as a code owner May 1, 2025 22:48
@github-actions github-actions bot added build Compilation issues examples python python script changes devops improvements to build systems and github actions labels May 1, 2025
@ngxson
Copy link
Collaborator

ngxson commented May 2, 2025

I think we also need to add LLAMA_BUILD_TOOLS=NO for certain CI, dockerfiles and some build scripts for android/mac/ios

Btw I think it's a good opportunity to change the name llava --> mtmd, I can do that in another PR

@github-actions github-actions bot added documentation Improvements or additions to documentation script Script related testing Everything test related labels May 2, 2025
@ngxson
Copy link
Collaborator

ngxson commented May 2, 2025

Btw, I think this could be a breaking change since some people can have automate script to pull things from certain places of llama.cpp ; for example, downstream projects like LM Studio and ollama pulls the llava source code, which is now moved to the new place.

@ggerganov I think making a comm like the last time we move the project from ggerganov --> ggml-org should be enough, WDYT?

@github-actions github-actions bot added the server label May 2, 2025
@ngxson
Copy link
Collaborator

ngxson commented May 2, 2025

@slaren the web ui may fails to build on windows (I suspect because the node does not use zlib windows? But not sure).

I'll push a build from macos rn, which should work

@slaren
Copy link
Member Author

slaren commented May 2, 2025

I am on WSL, which is a Linux VM, so it should work. I was wondering if it is due to the node version? I have 19.9.0 from the ubuntu repository.

@ngxson
Copy link
Collaborator

ngxson commented May 2, 2025

I never tested with node v19, but yeah that could be the reason.

A fix could be to use a gzip implementation in pure JS, which will be more deterministic.

@slaren slaren merged commit 1d36b36 into master May 2, 2025
10 of 11 checks passed
@slaren slaren deleted the sl/move-tools branch May 2, 2025 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Compilation issues devops improvements to build systems and github actions documentation Improvements or additions to documentation examples python python script changes script Script related server testing Everything test related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants