Skip to content

Commit 554fb40

Browse files
authored
Update migration guide - generation config (#42470)
* update nit * ig doc builder is complaining about this unclosed code block? * ? * ?? * then let's just not have another embedded code block in python block
1 parent b54f790 commit 554fb40

File tree

2 files changed

+7
-7
lines changed

2 files changed

+7
-7
lines changed

MIGRATION_GUIDE_V5.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,7 @@ model_4bit = AutoModelForCausalLM.from_pretrained(
330330
- It is no longer possible to load a config class from a URL file. Configs must be loaded from either a local path or a repo on the Hub. See [#42383](https://github.com/huggingface/transformers/pull/42383).
331331
- All parameters for configuring model's rotary embedding are now stored under `mode.rope_parameters`, including the `rope_theta` and `rope_type`. Model's `config.rope_parameters` is a simple dictionaty in most cases, and can also be a nested dict in special cases (i.e. Gemma3 and ModernBert) with different rope parameterization for each layer type. See [#39847](https://github.com/huggingface/transformers/pull/39847)
332332
- Qwen-VL family configuration is in a nested format and trying to access keys directly will throw an error (e.g. `config.vocab_size`). Users are expected to access keys from their respective sub-configs (`config.text_config.vocab_size`).
333+
- Configurations of non-generative models (any model that doesn't call `model.generate()`) will no longer have a `generation_config` and `model.config.generation_config` will throw an attribute error.
333334

334335
## Processing
335336

src/transformers/models/cohere/tokenization_cohere.py

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -280,8 +280,8 @@ def apply_tool_use_template(
280280
Examples:
281281
282282
```python
283-
tokenizer = CohereTokenizer.from_pretrained("CohereForAI/c4ai-command-r-v01")
284-
tools = [
283+
>> tokenizer = CohereTokenizer.from_pretrained("CohereForAI/c4ai-command-r-v01")
284+
>> tools = [
285285
{
286286
"name": "internet_search",
287287
"description": "Returns a list of relevant document snippets for a textual query retrieved from the internet",
@@ -299,16 +299,15 @@ def apply_tool_use_template(
299299
"parameter_definitions": {},
300300
},
301301
]
302-
conversation = [
302+
>> conversation = [
303303
{"role": "user", "content": "Whats the biggest penguin in the world?"},
304304
]
305-
# Render the prompt, ready for user to inspect, or for input into the model
306-
prompt = tokenizer.apply_tool_use_template(conversation, tools=tools, tokenize=False, add_generation_prompt=True)
307-
print(prompt)
305+
>> # Render the prompt, ready for user to inspect, or for input into the model
306+
>> prompt = tokenizer.apply_tool_use_template(conversation, tools=tools, tokenize=False, add_generation_prompt=True)
307+
>> print(prompt)
308308
>> inputs = tokenizer.encode(grounded_generation_prompt, add_special_tokens=False, return_tensors='pt')
309309
>> outputs = model.generate(inputs, max_new_tokens=128)
310310
>> print(tokenizer.decode(outputs[0]))
311-
Action: ```json
312311
[
313312
{
314313
"tool_name": "internet_search",

0 commit comments

Comments
 (0)