Skip to content

Conversation

@mgoin
Copy link
Member

@mgoin mgoin commented Sep 17, 2025

Purpose

The Kernels Attention Test job didn't run when #24761 landed

https://buildkite.com/vllm/ci/builds/31128/steps/canvas?jid=019957f9-c996-474a-8f31-03ce4d94f5f5

[2025-09-17T15:02:08Z] FAILED kernels/attention/test_attention_selector.py::test_invalid_env[True] - assert "Invalid attention backend: 'INVALID'" in "Invalid value 'INVALID' for VLLM_ATTENTION_BACKEND. Valid options: ['FLASH_ATTN', 'FLASH_ATTN_VLLM_V1', 'TRITON_ATTN_VLLM_V1', 'XFORMERS', 'ROCM_FLASH', 'ROCM_AITER_MLA', 'ROCM_AITER_MLA_VLLM_V1', 'ROCM_AITER_FA', 'TORCH_SDPA', 'TORCH_SDPA_VLLM_V1', 'FLASHINFER', 'FLASHINFER_VLLM_V1', 'FLASHINFER_MLA', 'TRITON_MLA', 'TRITON_MLA_VLLM_V1', 'CUTLASS_MLA', 'FLASHMLA', 'FLASHMLA_VLLM_V1', 'FLASH_ATTN_MLA', 'PALLAS', 'PALLAS_VLLM_V1', 'IPEX', 'DUAL_CHUNK_FLASH_ATTN', 'DIFFERENTIAL_FLASH_ATTN', 'NO_ATTENTION', 'FLEX_ATTENTION', 'TREE_ATTN', 'XFORMERS_VLLM_V1']."
[2025-09-17T15:02:08Z]  +  where "Invalid value 'INVALID' for VLLM_ATTENTION_BACKEND. Valid options: ['FLASH_ATTN', 'FLASH_ATTN_VLLM_V1', 'TRITON_ATTN_VLLM_V1', 'XFORMERS', 'ROCM_FLASH', 'ROCM_AITER_MLA', 'ROCM_AITER_MLA_VLLM_V1', 'ROCM_AITER_FA', 'TORCH_SDPA', 'TORCH_SDPA_VLLM_V1', 'FLASHINFER', 'FLASHINFER_VLLM_V1', 'FLASHINFER_MLA', 'TRITON_MLA', 'TRITON_MLA_VLLM_V1', 'CUTLASS_MLA', 'FLASHMLA', 'FLASHMLA_VLLM_V1', 'FLASH_ATTN_MLA', 'PALLAS', 'PALLAS_VLLM_V1', 'IPEX', 'DUAL_CHUNK_FLASH_ATTN', 'DIFFERENTIAL_FLASH_ATTN', 'NO_ATTENTION', 'FLEX_ATTENTION', 'TREE_ATTN', 'XFORMERS_VLLM_V1']." = str(ValueError("Invalid value 'INVALID' for VLLM_ATTENTION_BACKEND. Valid options: ['FLASH_ATTN', 'FLASH_ATTN_VLLM_V1', 'TRITON_ATTN_VLLM_V1', 'XFORMERS', 'ROCM_FLASH', 'ROCM_AITER_MLA', 'ROCM_AITER_MLA_VLLM_V1', 'ROCM_AITER_FA', 'TORCH_SDPA', 'TORCH_SDPA_VLLM_V1', 'FLASHINFER', 'FLASHINFER_VLLM_V1', 'FLASHINFER_MLA', 'TRITON_MLA', 'TRITON_MLA_VLLM_V1', 'CUTLASS_MLA', 'FLASHMLA', 'FLASHMLA_VLLM_V1', 'FLASH_ATTN_MLA', 'PALLAS', 'PALLAS_VLLM_V1', 'IPEX', 'DUAL_CHUNK_FLASH_ATTN', 'DIFFERENTIAL_FLASH_ATTN', 'NO_ATTENTION', 'FLEX_ATTENTION', 'TREE_ATTN', 'XFORMERS_VLLM_V1']."))
[2025-09-17T15:02:08Z]  +    where ValueError("Invalid value 'INVALID' for VLLM_ATTENTION_BACKEND. Valid options: ['FLASH_ATTN', 'FLASH_ATTN_VLLM_V1', 'TRITON_ATTN_VLLM_V1', 'XFORMERS', 'ROCM_FLASH', 'ROCM_AITER_MLA', 'ROCM_AITER_MLA_VLLM_V1', 'ROCM_AITER_FA', 'TORCH_SDPA', 'TORCH_SDPA_VLLM_V1', 'FLASHINFER', 'FLASHINFER_VLLM_V1', 'FLASHINFER_MLA', 'TRITON_MLA', 'TRITON_MLA_VLLM_V1', 'CUTLASS_MLA', 'FLASHMLA', 'FLASHMLA_VLLM_V1', 'FLASH_ATTN_MLA', 'PALLAS', 'PALLAS_VLLM_V1', 'IPEX', 'DUAL_CHUNK_FLASH_ATTN', 'DIFFERENTIAL_FLASH_ATTN', 'NO_ATTENTION', 'FLEX_ATTENTION', 'TREE_ATTN', 'XFORMERS_VLLM_V1'].") = <ExceptionInfo ValueError("Invalid value 'INVALID' for VLLM_ATTENTION_BACKEND. Valid options: ['FLASH_ATTN', 'FLASH_ATTN_VLLM_V1', 'T...UAL_CHUNK_FLASH_ATTN', 'DIFFERENTIAL_FLASH_ATTN', 'NO_ATTENTION', 'FLEX_ATTENTION', 'TREE_ATTN', 'XFORMERS_VLLM_V1'].") tblen=5>.value

Test Plan

Test Result

Tested locally


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mgoin mgoin added bug Something isn't working ci-failure Issue about an unexpected test failure in CI labels Sep 17, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a CI bugfix that updates an assertion in test_invalid_env to match a changed error message. The test verifies that setting an invalid attention backend via an environment variable raises a ValueError. The change updates the expected error message substring from "Invalid attention backend: 'INVALID'" to "Invalid value 'INVALID'". This change appears to be correct for the purpose of fixing the failing test. The new error message is less descriptive, which could be a minor regression in user experience, but the change in the test itself is sound. I did not find any issues with high or critical severity in this change.

@mgoin
Copy link
Member Author

mgoin commented Sep 17, 2025

Force merging to fix CI. Tested locally and precommit passes

@simon-mo simon-mo merged commit 087c6ff into vllm-project:main Sep 17, 2025
11 of 15 checks passed
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci-failure Issue about an unexpected test failure in CI

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants