-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Labels
models[Component] Issues related to model support[Component] Issues related to model support
Milestone
Description
Missing Thoughts and Unable to Token Stream with Claude Models
To leverage models put forward by Anthropic through model garden, we utilise the anthropic_llm.py where Claude models are defined. However, this does not allow for token streaming or forwarding reasoning tokens to the UI.
Solution
Implement:
- parsing Google GenAI's Thinking Config into Anthropic's Thinking Config to be passed along to anthropic API
- processing the anthropic API messages into response parts compatible with ADK
- allow for custom configurations (e.g., beta headers) in Anthropic API
Alternatives
- LiteLLM to process thinking tokens (Thought / Reasoning messages missing in events #2982)
Proposed Fix
PR created #3070
Metadata
Metadata
Assignees
Labels
models[Component] Issues related to model support[Component] Issues related to model support