Add support for models via LiteLLM proxy server

[haroon0x]

Is your feature request related to a problem? Please describe.
I’d like to add support for OpenAI models using the LiteLLM simple proxy. This will expand the model compatibility of adk-java to include OpenAI endpoints via a standardized proxy interface.

Describe the solution you'd like
-The plan is to use Java's HTTP client to call the proxy endpoint and integrate the response into the existing ADK pipeline.

[haroon0x]

@cornellgit opened this to track OpenAI model support using LiteLLM.
Let me know if you have any suggestions before I start.

[cornellgit]

I don't think they have any java sdk, therefore

• https://docs.litellm.ai/#litellm-proxy-server-llm-gateway is probably needed in the setup.
• https://litellm-api.up.railway.app/#/ seems like are all REST endpoint that are compatible with the original SDK by overriding the end point location to local, so you might be able to use the Java version https://platform.openai.com/docs/libraries#install-an-official-sdk
• Would be helpful to add a few example for other models too, since that's where LiteLLM is most helpful
• Some documentation to help users to run the setup e2e would be helpufl

For OAI model, it's frequently used, I'd suggest instead of going through LiteLLM route, try branch the code and adjust it to OAI. https://github.com/google/adk-java/blob/490f054abce6b857b5fec9022cd66646c108f213/core/src/main/java/com/google/adk/models/Claude.java

[vorburger]

I don't think they have any java sdk, therefore

BerriAI/litellm#11276

[manojkumarredbus]

One can use this for Ollama, it supports both function call and streaming, and currently being worked upon .
https://github.com/redbus-labs/adk-java/blob/main/core/src/main/java/com/google/adk/models/OllamaBaseLM.java

Also an Azure OAI compatible one available custom built , but one can port it to OAI compatible by removing additional auth code

https://github.com/redbus-labs/adk-java/blob/main/core/src/main/java/com/google/adk/models/RedbusADG.java

[vorburger]

Our LangChain4j connector in contrib/ might help here - have you seen and tried it out?

I'll mark this issue as pending your feedback. We'll close it if we don't hear back from you within ~1 month.

[julienba]

I found this issue when I tried to integrate a Java equivalent of LiteLLM and realized it was supported only for Python.

I managed to get it to work with Mistral using com.google.adk/google-adk-langchain4j. However, I'm losing the tokens used (the equivalent of the Gemini usageMetadata). Is there a way to get the metadata out of LLM integrated with lang4j?

Edit: There is already a PR for it #623!