[Bugfix] Fix feature size calculation for LLaVA-NeXT

[DarkLight1337]

This PR primarily updates our code to be consistent with huggingface/transformers#32314.

The feature size calculation has also been updated to no longer depend on the floating-point error in CUDA for some specific image resolutions, e.g.:

# original_height = 183, original_width = 488, height = 24
>>> int(183 * torch.tensor(24).cuda() / 488)
8
>>> int(183 * torch.tensor(24).cpu() / 488)
9

This fixes a bug in which the number of image placeholder tokens becomes incorrect for such edge cases when the model is run on other devices such as CPU.

cc @xwjiang2010

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

• Comment /ready on the PR
• Add ready label to the PR
• Enable auto-merge.

🚀

[ywang96]

LGTM but should we wait for a new transformers release then merge this with a version bump? It looks like that PR hasn't been released yet.

[ywang96]

Why do we no longer use BatchedTensors?

[DarkLight1337]

I updated the definition in #6836 but forgot to change the model files.

[DarkLight1337]

LGTM but should we wait for a new transformers release then merge this with a version bump? It looks like that PR hasn't been released yet.

It's an internal change so it shouldn't break anything. I think we can do this in parallel with transformers development.

[ywang96]

LGTM but should we wait for a new transformers release then merge this with a version bump? It looks like that PR hasn't been released yet.

It's an internal change so it shouldn't break anything. I think we can do this in parallel with transformers development.

I'm only worried about the model result consistency, but I guess since we're developing on the main branch it shouldn't be a big deal.

[DarkLight1337]

LGTM but should we wait for a new transformers release then merge this with a version bump? It looks like that PR hasn't been released yet.

It's an internal change so it shouldn't break anything. I think we can do this in parallel with transformers development.

I'm only worried about the model result consistency, but I guess since we're developing on the main branch it shouldn't be a big deal.

Yeah, either way there will be a period of inconsistency as we can't time our release to be at the same time as transformers.

[xwjiang2010]

LGTM, thanks for the fix!