[V0][V1][Core] Add outlines integration for V1, and update V0 integration. #15975

unaidedelf8777 · 2025-04-03T00:44:53Z

Adds outlines as a guided decoding backend for V1, and updates the integration for V0.

The aim of this is three fold:

Remove the dependency on outlines, and only use outlines_core
performance gains for V0 using the write_mask_into method on Guide to write a bitmask in-place for use in logits masking.
outlines backend for V1

Because the dependency on outlines will be removed, support for grammar based decoding with the outlines backend will also be removed (CFG classes reside in the outlines package)

cc @aarnphm

github-actions · 2025-04-03T00:45:05Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

unaidedelf8777 · 2025-04-03T00:49:54Z

NOTE: Can't be merged until next version of outlines_core is released.

russellb · 2025-04-07T20:04:36Z

Thank you for the PR! I will review it this week.

aarnphm

I reviewed the v0 code path. One ask is to add tests for this for disabling cache path.

And we should update the requirements/common.txt to the lowest version of outlines-core supported.

vllm/model_executor/guided_decoding/outlines_logits_processors.py

mergify · 2025-04-11T01:24:31Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @unaidedelf8777.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

unaidedelf8777 · 2025-04-13T17:54:51Z

@russellb @aarnphm All code is done. If you guys approve of it I’ll go ahead and clean up all the linter complaints, and then it should be ready.

Also outlines-core update has been pushed to pypi and pinned here (v0.2.9)

aarnphm

First round of review. A few things needs to be addressed here. but great progress so far.

vllm/model_executor/guided_decoding/outlines_decoding.py

vllm/model_executor/guided_decoding/outlines_logits_processors.py

vllm/v1/structured_output/backend_outlines.py

mergify · 2025-04-18T23:44:25Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @unaidedelf8777.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Nathan Hoos <[email protected]>

unaidedelf8777 · 2025-06-22T15:04:15Z

@russellb sorry about the hold up, I had to go to Mexico for a family situation last week. V1 tests are added and passing. The failing speculative decoding and quantization tests are currently failing on main according to this ci run. The LoRA test which fails seems to be unrelated to anything modified here.

aarnphm

Thanks for all the hard work and perseverance! The CI failure is not relevant and will be fixed separately.

but still will wait for @russellb to take a look once he's back

russellb · 2025-07-08T18:01:33Z

Apologies for the delay. Let's see if CI passes now if you merge from main.

…egration

Signed-off-by: Nathan Hoos <[email protected]>

russellb

We've talked about some next steps, but I don't want to risk you having to deal with large conflicts again. Some things that would be good to see:

Some benchmarking so we know how this compares to the other backends and which use cases it does best with.
Updates to docs/

Thank you for all of the hard work and diligence on this PR!

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: Patrick von Platen <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: avigny <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: Paul Pak <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: Diego-Castan <[email protected]>

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

unaidedelf8777 requested review from mgoin, russellb, WoosukKwon, robertgshaw2-redhat, njhill, ywang96, comaniac and alexm-redhat as code owners April 3, 2025 00:44

unaidedelf8777 changed the title ~~[V0][V1][Core] Add outlines integration for V1, and update V0 integration.~~ [V0][V1][Core] Add outlines integration for V1, and update V0 integration. [DO NOT MERGE] Apr 3, 2025

mergify bot added the v1 label Apr 3, 2025

unaidedelf8777 marked this pull request as draft April 3, 2025 00:46

unaidedelf8777 changed the title ~~[V0][V1][Core] Add outlines integration for V1, and update V0 integration. [DO NOT MERGE]~~ [V0][V1][Core] Add outlines integration for V1, and update V0 integration. Apr 3, 2025

hmellor mentioned this pull request Apr 3, 2025

Removes duplicate outlines processors #6900

Closed

mergify bot added the structured-output label Apr 4, 2025

aarnphm approved these changes Apr 7, 2025

View reviewed changes

vllm/model_executor/guided_decoding/outlines_logits_processors.py Outdated Show resolved Hide resolved

mergify bot added tpu Related to Google TPUs ci/build and removed tpu Related to Google TPUs labels Apr 9, 2025

mergify bot added needs-rebase and removed needs-rebase labels Apr 11, 2025

unaidedelf8777 marked this pull request as ready for review April 13, 2025 17:55

aarnphm requested changes Apr 14, 2025

View reviewed changes

mergify bot added needs-rebase and removed needs-rebase labels Apr 18, 2025

unaidedelf8777 added 2 commits June 22, 2025 00:52

make replacement_seq regex catch larger.

54c6075

Signed-off-by: Nathan Hoos <[email protected]>

fix mistral handling

b363dda

Signed-off-by: Nathan Hoos <[email protected]>

aarnphm requested a review from russellb June 23, 2025 19:54

aarnphm approved these changes Jun 23, 2025

View reviewed changes

Merge remote-tracking branch 'upstream/main' into update-outlines-int…

a435fee

…egration

unaidedelf8777 requested review from simon-mo, youkaichao, tlrmchlsmth, houseroad and hmellor as code owners July 8, 2025 18:11

add spdx headers for pre-commit

f294328

Signed-off-by: Nathan Hoos <[email protected]>

russellb approved these changes Jul 10, 2025

View reviewed changes

russellb merged commit d6902ce into vllm-project:main Jul 10, 2025
98 checks passed

github-project-automation bot moved this from In review to Done in Structured Output Jul 10, 2025

github-project-automation bot moved this to Done in Tool Calling Jul 10, 2025

Chen-zexi pushed a commit to Chen-zexi/vllm that referenced this pull request Jul 13, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

088a1b9

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

chaunceyjiang mentioned this pull request Jul 13, 2025

[Bugfix] Fix vLLM startup to avoid a hard dependency on Outlines #20875

Closed

LyrisZhong pushed a commit to LyrisZhong/vllm that referenced this pull request Jul 23, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

c788c1f

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

avigny pushed a commit to avigny/vllm that referenced this pull request Jul 31, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

411a659

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: avigny <[email protected]>

Pradyun92 pushed a commit to Pradyun92/vllm that referenced this pull request Aug 6, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

8169f08

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

fc3c6ea

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

4eaf356

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]> Signed-off-by: Paul Pak <[email protected]>

taneem-ibrahim pushed a commit to taneem-ibrahim/vllm that referenced this pull request Aug 14, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

ff44e12

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 27, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

f76b1a1

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

googlercolin pushed a commit to googlercolin/vllm that referenced this pull request Aug 29, 2025

[V0][V1][Core] Add outlines integration for V1, and update V0 integra…

9c01b5b

…tion. (vllm-project#15975) Signed-off-by: Nathan Hoos <[email protected]>

Uh oh!

[V0][V1][Core] Add outlines integration for V1, and update V0 integration. #15975

[V0][V1][Core] Add outlines integration for V1, and update V0 integration. #15975

Uh oh!

Conversation

unaidedelf8777 commented Apr 3, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 3, 2025

Uh oh!

unaidedelf8777 commented Apr 3, 2025

Uh oh!

russellb commented Apr 7, 2025

Uh oh!

aarnphm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mergify bot commented Apr 11, 2025

Uh oh!

unaidedelf8777 commented Apr 13, 2025

Uh oh!

aarnphm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Apr 18, 2025

Uh oh!

unaidedelf8777 commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aarnphm left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

russellb commented Jul 8, 2025

Uh oh!

russellb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

unaidedelf8777 commented Apr 3, 2025 •

edited by github-actions bot

Loading

unaidedelf8777 commented Jun 22, 2025 •

edited

Loading

aarnphm left a comment •

edited

Loading