[Bugfix] Fix Mistral3 spatial merge error

[mgoin]

FIX #16675

We just were not patching spatial_merge_size into the vision config in both of the places needed. This results in the dummy inputs having the right spatial_merge_size being applied but actual inferences were missing it, causing the shapes to mismatch during inference.

Here is where is it being applied already in vision.py

vllm/vllm/model_executor/models/vision.py

Lines 63 to 67 in 20e489e

	if isinstance(vision_config, PixtralVisionConfig):
	# Need to sneak in spatial_merge_size for Mistral3
	vision_config.spatial_merge_size = getattr(hf_config,
	"spatial_merge_size", 1)
	return PixtralHFEncoderInfo(vision_config)

With this PR, the example passes:

python ~/code/vllm/examples/offline_inference/vision_language.py -m mistral3 

Processed prompts: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:03<00:00,  1.23it/s, est. speed input: 2559.91 toks/s, output: 78.61 toks/s]
--------------------------------------------------
The image depicts a vibrant and picturesque scene featuring the iconic Tokyo Skytree, a tall broadcasting and observation tower located in Tokyo, Japan. The Skytree is framed by cherry blossom trees in full bloom, creating a beautiful, natural archway of pink flowers. The sky is clear and bright blue, enhancing
--------------------------------------------------
The image depicts a picturesque scene featuring cherry blossom trees in full bloom. The blossoms are predominantly pink and create a beautiful, almost tunnel-like effect with their branches framing the image. In the background, the iconic Tokyo Skytree, a tall broadcasting and observation tower located in Tokyo, Japan, is visible.
--------------------------------------------------
The image depicts a picturesque scene featuring the iconic Tokyo Skytree, a tall broadcasting and observation tower located in Tokyo, Japan. The tower is framed by a beautiful canopy of cherry blossom (sakura) trees in full bloom. The cherry blossoms are in various shades of pink, creating a vibrant and colorful
--------------------------------------------------
This image features a picturesque scene of cherry blossom trees in full bloom with their vibrant pink flowers. The trees frame the image, creating a natural archway that leads the viewer's eye towards the central focus of the image: the iconic Tokyo Skytree, a tall broadcasting and observation tower located in Tokyo, Japan
--------------------------------------------------

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[DarkLight1337]

Thanks for debugging and fixing!