Skip to content

Commit b8ba10d

Browse files
committed
Update helm chart README with pluginsCustomConfig flag
1 parent a3f25c0 commit b8ba10d

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

config/charts/inferencepool/README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,25 @@ Then apply it with:
4545
$ helm install vllm-llama3-8b-instruct ./config/charts/inferencepool -f values.yaml
4646
```
4747

48+
### Install with Custom EPP Plugins Configuration
49+
50+
To set custom EPP plugin config, you can pass it as an inline yaml. For example:
51+
52+
```yaml
53+
pluginsCustomConfig:
54+
custom-plugins.yaml: |
55+
apiVersion: inference.networking.x-k8s.io/v1alpha1
56+
kind: EndpointPickerConfig
57+
plugins:
58+
- type: custom-scorer
59+
parameters:
60+
custom-threshold: 64
61+
schedulingProfiles:
62+
- name: default
63+
plugins:
64+
- pluginRef: custom-scorer
65+
```
66+
4867
### Install with Additional Ports
4968
5069
To expose additional ports (e.g., for ZMQ), you can define them in the `values.yaml` file:
@@ -124,6 +143,7 @@ The following table list the configurable parameters of the chart.
124143
| `inferenceExtension.extraServicePorts` | List of additional service ports to expose. Defaults to `[]`. |
125144
| `inferenceExtension.flags` | List of flags which are passed through to endpoint picker. Example flags, enable-pprof, grpc-port etc. Refer [runner.go](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/cmd/epp/runner/runner.go) for complete list. |
126145
| `inferenceExtension.flags.has-enable-leader-election` | Enable leader election for high availability. When enabled, only one EPP pod (the leader) will be ready to serve traffic. |
146+
| `inferenceExtension.pluginsCustomConfig` | Custom config that is passed to EPP as inline yaml. |
127147
| `provider.name` | Name of the Inference Gateway implementation being used. Possible values: `gke`. Defaults to `none`. |
128148

129149
## Notes

0 commit comments

Comments
 (0)