Skip to content

Conversation

ArthurBook
Copy link
Contributor

Why are these changes needed?

vLLM v1 supports xgrammar and guidance since v0.8.2

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@ArthurBook ArthurBook requested a review from a team as a code owner May 9, 2025 17:27
@kouroshHakha
Copy link
Contributor

@ArthurBook thanks, have you confirmed that guided decoding with batch llm works for v1 as well? If so what vllm version have you tried it with?

@kouroshHakha kouroshHakha added the go add ONLY when ready to merge, run all tests label May 9, 2025
@kouroshHakha
Copy link
Contributor

kicking off release tests to make sure there is unseen issue: https://buildkite.com/ray-project/release/builds/41125

@lk-chen lk-chen changed the title fix: remove-no-longer needed guided decoding vllm v0 constraint [data][llm] fix: remove-no-longer needed guided decoding vllm v0 constraint May 9, 2025
@ArthurBook
Copy link
Contributor Author

Hi @kouroshHakha.
Runs fine with vLLM==0.8.5 + xgrammar==0.1.18 backend using vLLMEngineProcessorConfig with engine_args.guided_decoding_backend = "xgrammar".

@kouroshHakha kouroshHakha merged commit 86c0958 into ray-project:master May 9, 2025
5 checks passed
@ArthurBook ArthurBook deleted the remove-v0-guided-decoding-constraint branch May 9, 2025 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-backlog go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants