This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Implement auto-detection for clients #830

Closed

jhrozek opened this issue Jan 29, 2025 · 1 comment

Assignees

Contributor

jhrozek commented Jan 29, 2025

To centralize the ad-hoc client logic, let's start by adding auto-detection module that makes it easy to add new clients with a way to detect the client and allows us to search for places that need special handling in the codebase.

This will be followed up on by another issue that centralizes the special handling into the client code.

jhrozek self-assigned this

github-actions bot added the needs-triage label

jhrozek removed the needs-triage label

jhrozek mentioned this issue

Implement an interface to centralize the client differences #831

Open

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Add client detector interface

5b13645

Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Use the client type when streaming the data to the client, not when e…

82810d0

…xecuting completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Use the client type when instantiating and running provider pipelines

7e4474e

Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Remove get_tool_name_from_messages

568fe01

This was superseded by using the client enum.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Remove the is_copilot flag in favor of using the autodetected client

c6cbc41

In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Add client detector interface

6e0235f

Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Use the client type when streaming the data to the client, not when e…

2b20e3f

…xecuting completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Use the client type when instantiating and running provider pipelines

c1f8920

Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Remove get_tool_name_from_messages

1ac7e45

This was superseded by using the client enum.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Remove the is_copilot flag in favor of using the autodetected client

53c3316

In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Add client detector interface

963348e

Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Use the client type when streaming the data to the client, not when e…

29f143b

…xecuting completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Use the client type when instantiating and running provider pipelines

7eae7e1

Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Remove get_tool_name_from_messages

ecaee23

This was superseded by using the client enum.

Related: stacklok#830

jhrozek added a commit to jhrozek/codegate-open that referenced this issue


          Remove the is_copilot flag in favor of using the autodetected client

f3bb1a9

In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: stacklok#830

jhrozek added a commit that referenced this issue


          Use the client type when instantiating and running provider pipelines

ec96a31

Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: #830

jhrozek added a commit that referenced this issue


          Remove get_tool_name_from_messages

02c3347

This was superseded by using the client enum.

Related: #830

jhrozek added a commit that referenced this issue


          Remove the is_copilot flag in favor of using the autodetected client

6d31972

In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: #830

jhrozek added a commit that referenced this issue


          Add client detector interface

ea467c4

Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: #830

jhrozek added a commit that referenced this issue


          Use the client type when streaming the data to the client, not when e…

c4ada65

…xecuting completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: #830

jhrozek added a commit that referenced this issue


          Use the client type when instantiating and running provider pipelines

dc3b129

Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: #830

jhrozek added a commit that referenced this issue


          Remove get_tool_name_from_messages

8b227a7

This was superseded by using the client enum.

Related: #830

jhrozek added a commit that referenced this issue


          Remove the is_copilot flag in favor of using the autodetected client

d235c60

In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: #830

jhrozek added a commit that referenced this issue


          Add client detection (#832)

* Add client detector interface

Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: #830

* Use the client type when streaming the data to the client, not when executing completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: #830

* Use the client type when instantiating and running provider pipelines

Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: #830

* Remove get_tool_name_from_messages

This was superseded by using the client enum.

Related: #830

* Remove the is_copilot flag in favor of using the autodetected client

In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: #830

Contributor Author

jhrozek commented Feb 4, 2025

This was done, probably not referenced correctly in the PR.

jhrozek closed this as completed

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.