Skip to content

Conversation

@ngxson
Copy link

@ngxson ngxson commented Dec 25, 2025

ref comment: ggml-org#18334 (comment)

TL;DR:

  • Completely removed websocket
  • Replaced with SSE + HTTP POST request as mentioned earlier
  • Transport served under /mcp endpoint (GET + POST)
  • Use a custom transport class on webui to bypass the auth logic - we are running on the same host with the webui, not an external server, there is no need for CORS or auth
  • The code can be reduced even further

Benefits are:

  • Users already serving llama.cpp via reverse proxy have don't have to change anything - it just works
  • SSL is automatically handled by httplib
  • C++ code is much simpler than websocket

Tested with python tool:

image

@ochafik
Copy link
Owner

ochafik commented Dec 25, 2025

Thanks @ngxson !!
For SSE we'd need to proxy get /message too.

What issues did you face w/ the stock transports / why the new one?

Also we should be able to add auth headers (if needed for server started w/ --api-key) by giving requestInit.headers to the transport ctor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants