Models with multiple chat templates #6484
Description
Hi all, Matt from Hugging Face here. Just to let you know, we've made a modification to chat templates. As of Transformers v4.39, the tokenizer chat template field can now be a list containing multiple named templates. For compatibility reasons, the dict is converted to a list of {"name": "...", "template": "..."} dicts for serialization in tokenizer_config.json, but we convert it to a single dict when we load it in the tokenizer itself.
We did this to support Command-R and Command-R+, as they used separate templates for general LLM use, tool-assisted generation, and RAG. Right now, this is hardcoded in the tokenization.py file for Command-R, but we will be moving it into the model repos itself very soon - a repo PR is already open here, and we'll probably open one for Command-R+ too.
You can see how we're handling this here, but tl;dr if there are multiple templates, the user can request a template by name. If they don't request a name, then we use the one with the name "default" if that exists, or else we throw an error if there is no "default" template.
I'm not sure how exactly the chat template is copied into GGUF files, but the conversion script might need a small update to make sure it can handle this!