How to apply routing Plugin for POST body parsing to route to different backend services and also use default plugins on service level

[frosk1]

Hey,

I wanted to ask if it is possible in Kong to apply a single entrypoint route that uses a POST body parsing custom plugin, which routes the traffic to the corresponding backend service. But still I would want to apply the default plugins on the backend services.

The following config yaml examplifies this:

_format_version: "3.0"
_transform: true

services:
  - name: gpt-4o
    host: gpt-4o.internal.cluster
    port: 80
    protocol: http
    plugins:
      - name: rate-limiting
        config:
          minute: 10
          policy: local

  - name: gpt-35
    host: gpt-35.internal.cluster
    port: 80
    protocol: http
    plugins:
      - name: rate-limiting
        config:
          minute: 50
          policy: local

  - name: dummy-service
    host: dummy.internal
    port: 80
    protocol: http

routes:
  - name: chat-route
    paths:
      - /v1/chat/completions
    strip_path: false
    service: dummy-service

plugins:
  - name: body-model-router
    route: chat-route

I would want Kong to

1. receive a request under /v1/chat/completions
2. apply the custom route plugin body-model-router and filter out the backend service by the POST body model parameter
3. define which service it should route to
4. apply the default plugins defined in the config for this service

Like a unified API Gateway for openai API specifics, that routes traffic based on the POST body model parameter to a specific backend.

I guess this comes down to:
Can kong.service.set_target() be used to reference the service object defined in the config with the service specific plugins? Or is there another way that the route plugin could then trigger the service plugins instead of directly setting upstream host port?

[frosk1]

Would this be the solution:

in the POST body parsing plugin use kong.service.request.set_path(path) instead of kong.service.set_target() and define a separate route for each service object?

_format_version: "3.0"
_transform: true

services:
  - name: gpt-4o
    host: gpt-4o.internal.cluster
    port: 80
    protocol: http
    plugins:
      - name: rate-limiting
        config:
          minute: 10
          policy: local

  - name: gpt-35
    host: gpt-35.internal.cluster
    port: 80
    protocol: http
    plugins:
      - name: rate-limiting
        config:
          minute: 50
          policy: local

  - name: dummy-service
    host: dummy.internal
    port: 80
    protocol: http

routes:
  - name: chat-route
    paths:
      - /v1/chat/completions
    strip_path: false
    service: dummy-service

  - name: gpt-4o-route
    paths:
      - /v1/chat/completions/gpt-4o
    strip_path: true
    service: gpt-4o

  - name: gpt-35-route
    paths:
      - /v1/chat/completions/gpt-3.5-turbo
    strip_path: true
    service: gpt-35

plugins:
  - name: body-model-router
    route: chat-route

[KianAnbarestani]

@frosk1
Yes, your approach is conceptually sound, but there's a limitation: using kong.service.set_target() only sets the upstream target (host/port), it does not switch the actual Kong service object, so service-specific plugins (like rate-limiting) won’t be executed.

To achieve what you want—route based on POST body content while still applying service-level plugins—you can:

1. Attach your custom body-model-router plugin to a single entrypoint route.
2. In the plugin, inspect the POST body and rewrite the request path using kong.service.request.set_path() based on the model parameter.
3. Define internal routes (e.g., /internal/gpt4o/chat) pointing to real services with their own plugins.
4. Let Kong re-evaluate the route → service → plugin chain accordingly.
   This ensures dynamic routing based on POST body while keeping default service plugins intact.

[frosk1]

Hey,

thx for the answer. So I think I maybe in the meantime have tried also this but not 100% sure.

After no solution I ended up letting the plugin send another http request to local host with the new route to trigger the default plugins, because I think I have read somewhere that nginx or open resty and thus Kong, does not provide a way where a request is coming in and a plugin would trigger a move again to the routing phase when already received the request so to say.

Not sure if I outline this correctly, but to make it more straightforward:

Is the only way to trigger default plugins under a route from another route, when an entrypoint route triggers a plugin which sends a new http request to localhost?

PS: I know that this will actually increase memory load by the factor of two, since Every request to the entrypoint will generate a new request with the exact same payload and everything.