Refactor sliding window configuration to Transformers best practice

[hmellor]

• Uses layer_types instead of sliding_window_pattern
• Removes custom interleaved_sliding_window as it should not be necessary

Unlike the last time I tried to remove sliding_window: list[int] (#18494 (comment)), this PR correctly maps Mistral format sliding window configuration to the new HF style sliding window configuration.

---

This PR should also fix the issues that everyone has been seeing with Gemma 3.

Prior to this PR, vLLM was unable to detect that some Gemma 3 models were interleaved sliding attention models. This is because vLLM relied on the presence of sliding_window_pattern in the Gemma3Config. In earlier versions of Transformers this field was present and it was either read directly from config.json or derived based on the known pattern in Gemma 3.

However, in newer versions of Transformers sliding_window_pattern has been replaced with layer_types. This means that there is no longer a derived sliding_window_pattern in Gemma3Config and it will only appear if it is explicitly included in config.json. As you can see from this sample of Gemma 3 config.jsons, it was not always included:

• present https://huggingface.co/google/gemma-3-1b-it/blob/main/config.json
• not present https://huggingface.co/google/gemma-3-4b-it/blob/main/config.json
• not present https://huggingface.co/google/gemma-3-12b-it/blob/main/config.json
• not present https://huggingface.co/google/gemma-3-27b-it/blob/main/config.json

Since these configs contained sliding_window and vLLM was unable to recognise that they were interleaved, it treated Gemma 3 as an all sliding attention model. The lack of full attention layers can explain the decrease in task performance people saw.

PR updates vLLM to rely on the newer layer_types field, so it should fix the Gemma 3 models that were not working properly in vLLM. #20541 is a related PR that did add layer_types support locally in vLLM's Gemma 3 modelling code, but this PR applies it globally to all of vLLM so new models (and the Transformers backend) shouldn't have this issue.

Fixes huggingface/transformers#40017
Fixes #15752
Fixes #17689
Fixes #20341
Fixes #22270
Fixes #22475

[gemini-code-assist]

Code Review

This pull request refactors the sliding window configuration to align with Transformers' best practices by using layer_types instead of custom attributes like sliding_window_pattern and interleaved_sliding_window. This is a positive change that simplifies the codebase. However, I've identified two critical issues. Firstly, there's an inconsistency in the string used to identify sliding attention layers ('sliding_attention' vs. 'sliding_window') between the main configuration file and the model implementations, which could lead to incorrect behavior. Secondly, the refactoring seems to have removed support for per-layer sliding window sizes when provided as a list, which could be a regression and cause runtime errors.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[hmellor]

(sorry everyone...)

[hmellor]

I have updated the description to detail the consequences this PR has on Gemma 3 models

[DarkLight1337]

Thanks for fixing!

[DarkLight1337]

Need to merge from main again