Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Add client detection #832

Merged
merged 5 commits into from
Feb 4, 2025
Merged

Add client detection #832

merged 5 commits into from
Feb 4, 2025

Conversation

jhrozek
Copy link
Contributor

@jhrozek jhrozek commented Jan 29, 2025

  • Add client detector interface - Adds a decorator that can be added to the FastAPI handlers and detect the client from a fallback mechanism, by the user-agent, by a specific header or by a matching word in the messages. At the moment, the clients are represented as a simple enum, but in follow-up patches they will be represented by classes that can perform the changes by an interface providing callbacks from the pipeline or other places that need client-specific behaviour.
  • Use the client type when streaming the data to the client, not when executing completion - We used to special-case ollama stream generation by passing the client type to the execute_completion. Instead, let's pass the client type to the place that needs special casing using the recently introduce client type enum.
  • Use the client type when instantiating and running provider pipelines - Instead of detecting the client type again when the pipeline is being processed, let's pass the client type on instantiating the pipeline instance as a constant and replace the hardcoded client strings by just using the constants.
  • Remove get_tool_name_from_messages - This was superseded by using the client enum.
  • Remove the is_copilot flag in favor of using the autodetected client - In the copilot provider, we can hardcode the client type to copilot when instantiating the pipelines.

aponcedeleonch
aponcedeleonch previously approved these changes Jan 31, 2025
Copy link
Contributor

@aponcedeleonch aponcedeleonch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!

@jhrozek
Copy link
Contributor Author

jhrozek commented Feb 3, 2025

I rebased the branch and fixed some smallish issues along the way. Ready for a next round.

@jhrozek
Copy link
Contributor Author

jhrozek commented Feb 3, 2025

Good guy integration test, catching my bugs.

@lukehinds
Copy link

Good work @jhrozek , just needs a rebase.

lukehinds
lukehinds previously approved these changes Feb 4, 2025
Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: #830
…xecuting completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: #830
Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: #830
This was superseded by using the client enum.

Related: #830
In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: #830
@jhrozek jhrozek merged commit 0599103 into main Feb 4, 2025
9 checks passed
@jhrozek jhrozek deleted the client_detector branch February 4, 2025 09:30
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants