[Misc] Replace os environ to monkeypatch in test suite #14516

t-sibiraj · 2025-03-09T09:58:34Z

Convert all os.environ(xxx) to monkeypatch.setenv in test suite

Submitting pull request again because I forget to sign my original pull request

Co-authored-by: Aaron Pham [email protected]
Signed-off-by: Aaron Pham [email protected]

github-actions · 2025-03-09T09:58:45Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

tests/basic_correctness/test_cumem.py

tests/test_utils.py

DarkLight1337 · 2025-03-09T15:12:01Z

Please fix the pre-commit errors

robertgshaw2-redhat · 2025-03-09T16:58:14Z

tests/models/decoder_only/language/test_gptq_marlin.py

you cant pass monkeypatch to a module level fixture

robertgshaw2-redhat · 2025-03-09T16:58:21Z

tests/models/decoder_only/vision_language/test_models.py

robertgshaw2-redhat · 2025-03-09T16:58:27Z

tests/models/decoder_only/vision_language/test_phi3v.py

robertgshaw2-redhat · 2025-03-09T20:06:04Z

I would suggest installing the pre-commit which will automatically sign off for you. Run ./format.sh locally and it will give you instructions

t-sibiraj · 2025-03-11T17:55:50Z

I would suggest installing the pre-commit which will automatically sign off for you. Run ./format.sh locally and it will give you instructions

okie will test locally first

tests/models/decoder_only/vision_language/test_models.py

Signed-off-by: sibi <[email protected]>

Co-authored-by: Cyrus Leung <[email protected]> Signed-off-by: sibi <[email protected]>

… variable removal Co-authored-by: Cyrus Leung <[email protected]> Signed-off-by: sibi <[email protected]>

Signed-off-by: sibi <[email protected]>

Co-authored-by: Cyrus Leung <[email protected]> Signed-off-by: sibi <[email protected]>

…ld push the changes and pass the pre-commit check Signed-off-by: sibi <[email protected]>

Signed-off-by: sibi <[email protected]>

t-sibiraj · 2025-03-15T14:47:50Z

@t-sibiraj you can apply this patch befor 4932bcd

14516.patch

Thank you so much I will apply the patch

aarnphm · 2025-03-15T16:41:48Z

hmm, can you add me as collaborator to your branch? I can help with expedite this PR and fix CI error :)

t-sibiraj · 2025-03-15T17:07:09Z

hmm, can you add me as collaborator to your branch? I can help with expedite this PR and fix CI error :)

Thank you I really appreciate your help. I have sent you an invite.

Signed-off-by: Aaron Pham <[email protected]>

aarnphm · 2025-03-16T16:20:24Z

@t-sibiraj can you add the following to the PR description?

Co-authored-by: Aaron Pham <[email protected]>
Signed-off-by: Aaron Pham <[email protected]>

Thanks

Signed-off-by: Aaron Pham <[email protected]>

DarkLight1337 · 2025-03-17T03:34:14Z

Done, and thanks for fixing this!

DarkLight1337 · 2025-03-17T03:35:48Z

The CI failures are unrelated so let's merge this

…14516) Signed-off-by: sibi <[email protected]> Signed-off-by: Aaron Pham <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Aaron Pham <[email protected]> Signed-off-by: DefTruth <[email protected]>

ProExpertProg · 2025-03-27T21:44:13Z

This PR broke the PyTorch Fullgraph tests:

[2025-03-16T17:00:51Z] Running 2 items in this shard: tests/compile/test_full_graph.py::test_full_graph[model_info0-2], tests/compile/test_full_graph.py::test_full_graph[model_info0-3]
--
  | [2025-03-16T17:00:51Z]
  | [2025-03-16T17:00:51Z] compile/test_full_graph.py::test_full_graph[model_info0-2] SKIPPED (...)
  | [2025-03-16T17:00:51Z] compile/test_full_graph.py::test_full_graph[model_info0-3] SKIPPED (...)

The fixture cannot be used in this simple way to generate an array of parameters

…14516) Signed-off-by: sibi <[email protected]> Signed-off-by: Aaron Pham <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Aaron Pham <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>

Pass pooling_metadata to pooler head in gritlm. This was broken by PR vllm-project#16331 broke gritlm. PR vllm-project#14516 broke gritlm tests due to changing xformers to flash_atnn Signed-off-by: Pooya Davoodi <[email protected]>

…14516) Signed-off-by: sibi <[email protected]> Signed-off-by: Aaron Pham <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Aaron Pham <[email protected]>

…14516) Signed-off-by: sibi <[email protected]> Signed-off-by: Aaron Pham <[email protected]> Co-authored-by: Cyrus Leung <[email protected]> Co-authored-by: Aaron Pham <[email protected]> Signed-off-by: Mu Huai <[email protected]>

t-sibiraj requested review from DarkLight1337, WoosukKwon, mgoin, russellb, tlrmchlsmth and ywang96 as code owners March 9, 2025 09:58

mergify bot added the v1 label Mar 9, 2025

DarkLight1337 reviewed Mar 9, 2025

View reviewed changes

tests/basic_correctness/test_cumem.py Outdated Show resolved Hide resolved

DarkLight1337 reviewed Mar 9, 2025

View reviewed changes

tests/test_utils.py Outdated Show resolved Hide resolved

DarkLight1337 approved these changes Mar 9, 2025

View reviewed changes

robertgshaw2-redhat reviewed Mar 9, 2025

View reviewed changes

tests/models/decoder_only/language/test_gptq_marlin.py Outdated

Copy link

Collaborator

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you cant pass monkeypatch to a module level fixture

robertgshaw2-redhat reviewed Mar 9, 2025

View reviewed changes

tests/models/decoder_only/vision_language/test_models.py Outdated

Copy link

Collaborator

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

robertgshaw2-redhat reviewed Mar 9, 2025

View reviewed changes

tests/models/decoder_only/vision_language/test_phi3v.py Outdated

Copy link

Collaborator

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

t-sibiraj force-pushed the replace-os-environ-monkeypatch branch from a6ade47 to 1e64dc0 Compare March 12, 2025 07:14

DarkLight1337 reviewed Mar 12, 2025

View reviewed changes

tests/models/decoder_only/vision_language/test_models.py Outdated Show resolved Hide resolved

DarkLight1337 enabled auto-merge (squash) March 12, 2025 08:10

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 12, 2025

t-sibiraj and others added 6 commits March 12, 2025 16:25

Replace os.environ with monkeypatch in test suite

4e69ecc

Signed-off-by: sibi <[email protected]>

Replace os.environ with monkeypatch in test suite

0f9ba59

Signed-off-by: sibi <[email protected]>

Update tests/basic_correctness/test_cumem.py

44c73bb

Co-authored-by: Cyrus Leung <[email protected]> Signed-off-by: sibi <[email protected]>

fix: Use raising=True in monkeypatch.delenv for safer environment…

5f28f2d

… variable removal Co-authored-by: Cyrus Leung <[email protected]> Signed-off-by: sibi <[email protected]>

doing a signed commit

8436a5f

Signed-off-by: sibi <[email protected]>

remove unnecessary spacing

b791b51

Co-authored-by: Cyrus Leung <[email protected]> Signed-off-by: sibi <[email protected]>

auto-merge was automatically disabled March 12, 2025 08:26
Head branch was pushed to by a user without write access

t-sibiraj force-pushed the replace-os-environ-monkeypatch branch from e9c8188 to b791b51 Compare March 12, 2025 08:26

Intentionally added and removed a space to test this commit, so I cou…

6971e83

…ld push the changes and pass the pre-commit check Signed-off-by: sibi <[email protected]>

Apply patch before 4932bcd

7139925

Signed-off-by: sibi <[email protected]>

t-sibiraj force-pushed the replace-os-environ-monkeypatch branch from 4932bcd to 7139925 Compare March 15, 2025 14:47

aarnphm added 6 commits March 15, 2025 13:24

fix: correct tests

48b9bf4

Signed-off-by: Aaron Pham <[email protected]>

fix: correct types

bb8540c

Signed-off-by: Aaron Pham <[email protected]>

fix: ignore otel imports mypy

7430fc6

Signed-off-by: Aaron Pham <[email protected]>

fix: types

9191625

Signed-off-by: Aaron Pham <[email protected]>

fix: distributed tests with dispatch

e324fb2

Signed-off-by: Aaron Pham <[email protected]>

chore: fix tests

5266dff

Signed-off-by: Aaron Pham <[email protected]>

mergify bot added the ci/build label Mar 16, 2025

fix: precommit error

a46f271

Signed-off-by: Aaron Pham <[email protected]>

DarkLight1337 enabled auto-merge (squash) March 17, 2025 03:34

vllm-bot merged commit a73e183 into vllm-project:main Mar 17, 2025
55 of 58 checks passed

aarnphm deleted the replace-os-environ-monkeypatch branch March 17, 2025 04:53

ProExpertProg mentioned this pull request Mar 27, 2025

[Bugfix] [torch.compile] Add Dynamo metrics context during compilation #15639

Merged

pooyadavoodi mentioned this pull request Apr 15, 2025

[Bugfix] Fix broken GritLM model and tests (missing pooling_metadata) #16631

Merged

ckhordiasma mentioned this pull request Apr 17, 2025

[do not merge] pr test for nm changes into 2.20 red-hat-data-services/vllm#107

Closed

Uh oh!

[Misc] Replace os environ to monkeypatch in test suite #14516

[Misc] Replace os environ to monkeypatch in test suite #14516

Uh oh!

Conversation

t-sibiraj commented Mar 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 9, 2025

Uh oh!

Uh oh!

Uh oh!

DarkLight1337 commented Mar 9, 2025

Uh oh!

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

robertgshaw2-redhat commented Mar 9, 2025

Uh oh!

t-sibiraj commented Mar 11, 2025

Uh oh!

Uh oh!

t-sibiraj commented Mar 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aarnphm commented Mar 15, 2025

Uh oh!

t-sibiraj commented Mar 15, 2025

Uh oh!

aarnphm commented Mar 16, 2025

Uh oh!

DarkLight1337 commented Mar 17, 2025

Uh oh!

DarkLight1337 commented Mar 17, 2025

Uh oh!

Uh oh!

ProExpertProg commented Mar 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

t-sibiraj commented Mar 9, 2025 •

edited by github-actions bot

Loading

t-sibiraj commented Mar 15, 2025 •

edited

Loading