[Bugfix] Make dummy encoder prompt padding alternative and add missing warnings #16129
Conversation
Signed-off-by: Isotr0py <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
|
Can we add a test to avoid similar regressions? Preferably using Whisper so it can run in CI without OOM |
|
cc @tjohnson31415 as well |
Hmmm, the original issue is Mllama specific, because the Mllama prefill/decode implementation is a bit hacky. Let me see if we can add a processor test to simulate this issue rather than loading the full model... |
Signed-off-by: Isotr0py <[email protected]>
| @pytest.mark.parametrize("model_id", | ||
| ["meta-llama/Llama-3.2-11B-Vision-Instruct"]) | ||
| @pytest.mark.parametrize("max_model_len", [4096, 8192, 25600, 131072]) | ||
| @pytest.mark.parametrize("max_num_seqs", [1, 2, 8]) | ||
| def test_profiling( |
There was a problem hiding this comment.
Has confirmed this test is failing on main branch.
github-actions
bot
added
the
ready
| # Encoder-decoder multimodal models only support v0 | ||
| if total_len > seq_len: | ||
| # `max_num_batched_tokens` is defined by `SchedulerConfig` | ||
| logger.warning( |
There was a problem hiding this comment.
I think this warning should only get printed once. As it it, it is just too noisy
…g warnings (vllm-project#16129) Signed-off-by: Isotr0py <[email protected]>
…g warnings (vllm-project#16129) Signed-off-by: Isotr0py <[email protected]> Signed-off-by: Yang Wang <[email protected]>
…g warnings (vllm-project#16129) Signed-off-by: Isotr0py <[email protected]>
…g warnings (vllm-project#16129) Signed-off-by: Isotr0py <[email protected]> Signed-off-by: Mu Huai <[email protected]>
3 participants
FIX #13929