generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 180
Closed
Labels
kind/featureCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.
Description
What would you like to be added:
Allow defining a default base model in the lora syncer configuration
Why is this needed:
It is repetitive to list it in each list item in our case since we are mostly assuming a single base model per pool.
so instead of
apiVersion: v1
kind: ConfigMap
metadata:
name: vllm-llama2-7b-adapters
data:
configmap.yaml: |
vLLMLoRAConfig:
name: vllm-llama2-7b
port: 8000
ensureExist:
models:
- base-model: meta-llama/Llama-2-7b-hf
id: tweet-summary-1
source: vineetsharma/qlora-adapter-Llama-2-7b-hf-TweetSumm
- base-model: meta-llama/Llama-2-7b-hf
id: tweet-summary-2
source: mahimairaja/tweet-summarization-llama-2-finetuned
we should be able to define it as
apiVersion: v1
kind: ConfigMap
metadata:
name: vllm-llama2-7b-adapters
data:
configmap.yaml: |
vLLMLoRAConfig:
name: vllm-llama2-7b
port: 8000
defaultBaseModel: meta-llama/Llama-2-7b-hf
ensureExist:
models:
- id: tweet-summary-1
source: vineetsharma/qlora-adapter-Llama-2-7b-hf-TweetSumm
- id: tweet-summary-2
source: mahimairaja/tweet-summarization-llama-2-finetuned
Metadata
Metadata
Assignees
Labels
kind/featureCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.