Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9fe9994
feat: Add Qwen3 omni moe thinker
Sep 24, 2025
93efc39
update registry and models page
Sep 24, 2025
81fd24b
Merge branch 'main' into dev/qwen3-omni-moe
DarkLight1337 Sep 27, 2025
f0d057a
Update w.r.t. #16229
DarkLight1337 Sep 27, 2025
c3e15a6
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 3, 2025
d59ac08
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 3, 2025
0b24c98
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 3, 2025
087a936
remove attn mask
ywang96 Oct 3, 2025
8ffc26e
update
ywang96 Oct 4, 2025
fb1d82b
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 5, 2025
7f42fb0
fix backend import
ywang96 Oct 5, 2025
7408b9c
fix prompt update
ywang96 Oct 5, 2025
8e1f5aa
yapf
ywang96 Oct 5, 2025
3c44f89
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 5, 2025
a985baa
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 6, 2025
0525d27
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 8, 2025
7484970
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 9, 2025
3c6243b
cleanup
ywang96 Oct 9, 2025
b8ec4d6
fix
ywang96 Oct 9, 2025
4c749d1
add
ywang96 Oct 9, 2025
650855a
fix
ywang96 Oct 9, 2025
03e1310
fix mixed modality
ywang96 Oct 9, 2025
7796103
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 9, 2025
2dee5f6
remove unnecessary tensor creation
ywang96 Oct 9, 2025
a6cb680
add note
ywang96 Oct 9, 2025
b4137ab
simplify
ywang96 Oct 9, 2025
d82c17e
cleanup
ywang96 Oct 9, 2025
677412d
cleanup
ywang96 Oct 9, 2025
bb00572
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 9, 2025
24c2c95
add guard
ywang96 Oct 9, 2025
14c6903
update
ywang96 Oct 9, 2025
51366bd
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 9, 2025
dc17c61
Merge branch 'main' into dev/qwen3-omni-moe
ywang96 Oct 10, 2025
ae0c930
fix
ywang96 Oct 10, 2025
3ec407f
add qwen3-omni processor test
Isotr0py Oct 10, 2025
897245a
fix non audio_in_video update
Isotr0py Oct 10, 2025
2465f49
fix qwen3-omni processor test
Isotr0py Oct 10, 2025
5d7ddf7
fix registry
Isotr0py Oct 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/models/supported_models.md
Original file line number Diff line number Diff line change
Expand Up @@ -713,6 +713,7 @@ These models primarily accept the [`LLM.generate`](./generative_models.md#llmgen
| `Qwen2_5OmniThinkerForConditionalGeneration` | Qwen2.5-Omni | T + I<sup>E+</sup> + V<sup>E+</sup> + A<sup>+</sup> | `Qwen/Qwen2.5-Omni-3B`, `Qwen/Qwen2.5-Omni-7B` | ✅︎ | ✅︎ | ✅︎ |
| `Qwen3VLForConditionalGeneration` | Qwen3-VL | T + I<sup>E+</sup> + V<sup>E+</sup> | `Qwen/Qwen3-VL-4B-Instruct`, etc. | ✅︎ | ✅︎ | ✅︎ |
| `Qwen3VLMoeForConditionalGeneration` | Qwen3-VL-MOE | T + I<sup>E+</sup> + V<sup>E+</sup> | `Qwen/Qwen3-VL-30B-A3B-Instruct`, etc. | ✅︎ | ✅︎ | ✅︎ |
| `Qwen3OmniMoeThinkerForConditionalGeneration` | Qwen3-Omni | T + I<sup>E+</sup> + V<sup>E+</sup> + A<sup>+</sup> | `Qwen/Qwen3-Omni-30B-A3B-Instruct`, `Qwen/Qwen3-Omni-30B-A3B-Thinking` | ✅︎ | ✅︎ | ✅︎ |
| `RForConditionalGeneration` | R-VL-4B | T + I<sup>E+</sup> | `YannQi/R-4B` | | ✅︎ | ✅︎ |
| `SkyworkR1VChatModel` | Skywork-R1V-38B | T + I | `Skywork/Skywork-R1V-38B` | | ✅︎ | ✅︎ |
| `SmolVLMForConditionalGeneration` | SmolVLM2 | T + I | `SmolVLM2-2.2B-Instruct` | ✅︎ | | ✅︎ |
Expand Down Expand Up @@ -803,8 +804,7 @@ Some models are supported only via the [Transformers backend](#transformers). Th
Our PaliGemma implementations have the same problem as Gemma 3 (see above) for both V0 and V1.

!!! note
For Qwen2.5-Omni, reading audio from video pre-processing (`--mm-processor-kwargs '{"use_audio_in_video": true}'`)
is currently supported on V0 (but not V1), because overlapping modalities is not yet supported in V1.
For Qwen2.5-Omni and Qwen3-Omni, reading audio from video pre-processing (`--mm-processor-kwargs '{"use_audio_in_video": true}'`) is currently work in progress and not yet supported.

#### Transcription

Expand Down
1 change: 1 addition & 0 deletions tests/models/multimodal/processing/test_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,7 @@ def _test_processing_correctness_one(
"Qwen/Qwen2.5-Omni-3B",
"Qwen/Qwen3-VL-4B-Instruct",
"Qwen/Qwen3-VL-30B-A3B-Instruct",
"Qwen/Qwen3-Omni-30B-A3B-Instruct",
"YannQi/R-4B",
"Skywork/Skywork-R1V-38B",
"HuggingFaceTB/SmolVLM2-2.2B-Instruct",
Expand Down
5 changes: 5 additions & 0 deletions tests/models/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -772,6 +772,11 @@ def check_available_online(
min_transformers_version="4.57",
is_available_online=False,
),
"Qwen3OmniMoeForConditionalGeneration": _HfExamplesInfo(
"Qwen/Qwen3-Omni-30B-A3B-Instruct",
max_model_len=4096,
min_transformers_version="4.57",
),
"RForConditionalGeneration": _HfExamplesInfo("YannQi/R-4B", trust_remote_code=True),
"SkyworkR1VChatModel": _HfExamplesInfo(
"Skywork/Skywork-R1V-38B", trust_remote_code=True
Expand Down
Loading