What are the ways we can change the system prompt template?

[timothylimyl]

Hi, can you direct me to where can I edit the system prompts for chat models?

I tried vllm (editing the endpoint code of async def generate() with Zephyr-7B and the results in terms of speed and accuracy is pretty good but I think it will be even better if I can tweak the default system prompt to according to my task. Currently, I can only edit the prompts going into engine.generate(prompt....)

Thanks in advance.

[timothylimyl]

realised vllm side does not manage the prompts which is good, we can just build a prompt template wrapper over our prompt as we see fit.