This repository was archived by the owner on Jun 5, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 83
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
jhrozek
commented
Jan 29, 2025
- Add client detector interface - Adds a decorator that can be added to the FastAPI handlers and detect the client from a fallback mechanism, by the user-agent, by a specific header or by a matching word in the messages. At the moment, the clients are represented as a simple enum, but in follow-up patches they will be represented by classes that can perform the changes by an interface providing callbacks from the pipeline or other places that need client-specific behaviour.
- Use the client type when streaming the data to the client, not when executing completion - We used to special-case ollama stream generation by passing the client type to the execute_completion. Instead, let's pass the client type to the place that needs special casing using the recently introduce client type enum.
- Use the client type when instantiating and running provider pipelines - Instead of detecting the client type again when the pipeline is being processed, let's pass the client type on instantiating the pipeline instance as a constant and replace the hardcoded client strings by just using the constants.
- Remove get_tool_name_from_messages - This was superseded by using the client enum.
- Remove the is_copilot flag in favor of using the autodetected client - In the copilot provider, we can hardcode the client type to copilot when instantiating the pipelines.
c6cbc41
to
53c3316
Compare
aponcedeleonch
previously approved these changes
Jan 31, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing!
53c3316
to
f3bb1a9
Compare
I rebased the branch and fixed some smallish issues along the way. Ready for a next round. |
f3bb1a9
to
6d31972
Compare
Good guy integration test, catching my bugs. |
Good work @jhrozek , just needs a rebase. |
lukehinds
previously approved these changes
Feb 4, 2025
Adds a decorator that can be added to the FastAPI handlers and detect the client from a fallback mechanism, by the user-agent, by a specific header or by a matching word in the messages. At the moment, the clients are represented as a simple enum, but in follow-up patches they will be represented by classes that can perform the changes by an interface providing callbacks from the pipeline or other places that need client-specific behaviour. Related: #830
…xecuting completion We used to special-case ollama stream generation by passing the client type to the execute_completion. Instead, let's pass the client type to the place that needs special casing using the recently introduce client type enum. Related: #830
Instead of detecting the client type again when the pipeline is being processed, let's pass the client type on instantiating the pipeline instance as a constant and replace the hardcoded client strings by just using the constants. Related: #830
This was superseded by using the client enum. Related: #830
In the copilot provider, we can hardcode the client type to copilot when instantiating the pipelines. Related: #830
6d31972
to
d235c60
Compare
rdimitrov
approved these changes
Feb 4, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.