Skip to content

Enable support for adding chat templates to base models #437

@RobotSail

Description

@RobotSail

Today in order to train with an arbitrary model, its tokenizer must already have a chat template and an eos_token set.

But in if you want to train a base model with a chat template, you must use our hard-coded chat templates.

We want to implement the ability to give a base model an arbitrary chat template through the following methods:

  • specifying an existing instruct model to use as the target
  • providing some minimal metadata for the model to use (chat template + eos_token + pad token)

This will allow us to fully deprecate the hard-coded chat template system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions