[Structured Output][Refactor] Move apply_grammar_bitmask() method from ModelRunner to structured output utils
#21999
Conversation
shen-shanshan
requested review from
WoosukKwon,
alexm-redhat,
comaniac,
njhill,
robertgshaw2-redhat and
ywang96
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request removes a redundant null check for grammar_bitmask. This is a good code cleanup that improves maintainability by removing unnecessary code. The change is straightforward and relies on the caller performing the null check, as stated in the PR description.
|
CC: @russellb |
shen-shanshan
changed the title
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
shen-shanshan
changed the title
if check for bitmask in model runner to assert
shen-shanshan
changed the title
if check for bitmask in model runner to assertapply_grammar_bitmask() method from ModelRunner to StructuredOutputManager
shen-shanshan
changed the title
apply_grammar_bitmask() method from ModelRunner to StructuredOutputManagerapply_grammar_bitmask() method from ModelRunner to StructuredOutputManager
russellb
left a comment
russellb
left a comment
There was a problem hiding this comment.
The StructuredOutputManager isn't an object used by the model runner right now. How about just making it a utility function somewhere instead of hanging it off of that class? Maybe here https://github.com/vllm-project/vllm/blob/main/vllm/v1/structured_output/utils.py
Thanks for your suggestion! I will modify it later~ |
|
This pull request has merge conflicts that must be resolved before it can be |
shen-shanshan
changed the title
apply_grammar_bitmask() method from ModelRunner to StructuredOutputManagerapply_grammar_bitmask() method from ModelRunner to structured output utils
OK, no problem. After these fix PR have been merged, I will update this soon. |
|
@russellb Hello, all the fix PRs you mentioned have already been merged, and I have rebased on the latest code. 😃 |
github-actions
bot
added
the
ready
Head branch was pushed to by a user without write access
|
This pull request has merge conflicts that must be resolved before it can be |
…tput utils Signed-off-by: shen-shanshan <[email protected]>
Signed-off-by: shen-shanshan <[email protected]>
Signed-off-by: shen-shanshan <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]> Signed-off-by: charlifu <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]>
…rom `ModelRunner` to structured output utils (vllm-project#21999) Signed-off-by: shen-shanshan <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
4 participants
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.Purpose
Currently, the feature structured output is closely coupled with the
model_runner, and we need to implement duplicateapply_grammar_bitmask()method in differentmodel_runnerof each platform, e.g.,gpu_model_runner,npu_model_runner. Once there are changes have made in this method, we need to update the method in all kinds ofmodel_runnerto sync these changes.Thus, maybe it's better to move these structured output related code in
model_runnerto thestructured_outputmodule to make it clearer and more extensible.Test Plan
Test Result
(Optional) Documentation Update