Skip to content

Commit d6085ba

Browse files
WangErXiaolulmer
authored andcommitted
[Doc]add doc for Qwen models tool calling (vllm-project#14478)
Signed-off-by: WangErXiao <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>
1 parent 89f34e0 commit d6085ba

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

docs/source/features/tool_calling.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,15 @@ AI21's Jamba-1.5 models are supported.
209209

210210
Flags: `--tool-call-parser jamba`
211211

212+
### Qwen Models
213+
214+
For Qwen2.5, the chat template in tokenizer_config.json has already included support for the Hermes-style tool use. Therefore, you can use the `hermes` parser to enable tool calls for Qwen models. For more detailed information, please refer to the official [Qwen documentation](https://qwen.readthedocs.io/en/latest/framework/function_call.html#vllm)
215+
216+
* `Qwen/Qwen2.5-*`
217+
* `Qwen/QwQ-32B`
218+
219+
Flags: `--tool-call-parser hermes`
220+
212221
### Models with Pythonic Tool Calls (`pythonic`)
213222

214223
A growing number of models output a python list to represent tool calls instead of using JSON. This has the advantage of inherently supporting parallel tool calls and removing ambiguity around the JSON schema required for tool calls. The `pythonic` tool parser can support such models.

0 commit comments

Comments
 (0)