[models][deepseek][qwen2.5] Add support for Qwen2.5 and Deepseek-Distilled-Qwen models #40
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mikepapadim
changed the title
orionpapadakis
force-pushed
the
feat/qwen2
branch
from
36ddcd6 to
3dd3474
Compare
mikepapadim
changed the title
Member
|
Fixes #19 |
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for Qwen2.5 and Deepseek-Distilled-Qwen models to the LLaMA inference framework. It introduces new model types, loaders, and computation kernels to handle these model architectures with their specific requirements.
Key changes:
- Added new model types
QWEN_2andDEEPSEEK_R1_DISTILL_QWENwith corresponding configurations and state management - Implemented specialized TornadoVM computation kernels for Qwen2 models including bias addition operations
- Added automatic reasoning token injection for Deepseek-R1-Distill-Qwen models
Reviewed Changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| TransformerComputeKernelsLayered.java | Added addInPlace kernel for element-wise array addition |
| TornadoVMMasterPlan.java | Added Qwen2 planner support and refactored model type dispatching |
| Qwen2TornadoVMLayerPlanner.java | New TornadoVM layer planner for Qwen2 models with bias operations |
| Qwen3Tokenizer.java | Updated token display logic to include reasoning tokens |
| Qwen3.java | Added shouldAddBeginOfText override |
| Qwen2Configuration.java | New configuration record for Qwen2 models |
| Qwen2.java | Main Qwen2 model implementation with DeepSeek-R1 specific behavior |
| Phi3.java | Added shouldAddBeginOfText override |
| Qwen2ModelLoader.java | Model loader for Qwen2/DeepSeek models with bias weight handling |
| ModelLoader.java | Updated model type detection logic |
| ModelType.java | Added new model types and DeepSeek detection |
| Model.java | Added reasoning token injection for DeepSeek models |
| Qwen2TornadoWeights.java | TornadoVM weights implementation for Qwen2 |
| Qwen2StandardWeights.java | Standard weights implementation for Qwen2 |
| Qwen2State.java | State management for Qwen2 models |
| InferenceCore.java | Java inference implementation for Qwen2 |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
src/main/java/org/beehive/gpullama3/tornadovm/Qwen2TornadoVMLayerPlanner.java
Outdated
Show resolved
Hide resolved
| .task("qbias", TransformerComputeKernelsLayered::addInPlace, state.wrapQ, weights.q_biasLayered[layerIndex], config.dim()) | ||
| .task("kbias", TransformerComputeKernelsLayered::addInPlace, state.wrapK, weights.k_biasLayered[layerIndex], config.kvDim()) | ||
| .task("vbias", TransformerComputeKernelsLayered::addInPlace, state.wrapV, weights.v_biasLayered[layerIndex], config.kvDim()) | ||
| .task("rope", Qwen3Kernels::ropeRotation,context, state.positionHolder, state.wrapQ, state.wrapK, config.numberOfKeyValueHeads(), |
There was a problem hiding this comment.
Using Qwen3Kernels::ropeRotation for Qwen2 models may be incorrect. Verify that Qwen2 and Qwen3 use identical RoPE implementations, or create a Qwen2-specific RoPE kernel.
| /** | ||
| * Executes the forward pass of a LLaMA transformer model using TornadoVM acceleration. This method processes the transformer layers in sequence for a particular token position in the context | ||
| * Executes the forward pass of a LLaMA transformer model using TornadoVM acceleration. | ||
| *This method processes the transformer layers in sequence for a particular token position in the context |
There was a problem hiding this comment.
Missing space after the asterisk in the comment. Should be '* This method' instead of '*This method'.
|
|
||
| @Override | ||
| public int contextLengthModel() { | ||
| return contextLengthModel; |
There was a problem hiding this comment.
The method returns the field contextLengthModel but should return the parameter contextLengthModel() to match the interface contract. This creates infinite recursion.
| try (var ignored = Timer.log("Load " + modelName + " model")) { | ||
| // reuse method of Qwen3 | ||
| Vocabulary vocabulary = loadQwen3Vocabulary(metadata); | ||
| boolean isDeepSeekR1DistillQwen = "DeepSeek-R1-Distill-Qwen".equals(metadata.get("general.basename")); |
There was a problem hiding this comment.
The string literal 'DeepSeek-R1-Distill-Qwen' is duplicated on lines 42 and 49. Consider extracting it to a constant to avoid duplication.
mikepapadim
approved these changes
Aug 29, 2025
…yerPlanner.java Co-authored-by: Copilot <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.