Skip to content

Conversation

@sducouedic
Copy link
Collaborator

@sducouedic sducouedic commented Jul 14, 2025

Closes #305

The changes in upstream that are breaking tests with vLLM:main branch are twofolds:

  1. This PR introduces an import which is incompatible with torch 2.5.1
  2. Previously, when model_config.task was set to auto, the task could be automatically resolved in vLLM: after the post-init method of ModelConfig, the task field was automatically replaced from auto to .e.g. generate or embed). With the changes in This PR, the task in model_config, cannot be automatically resolved anymore when it is generate. This doesn't look like it is intended, but rather a side effect of adding new supported tasks. A cleaner (but longer) way would be to do a PR to vLLM to fix that and reintroduce the automatic resolution of auto task field.

This PR addresses both of the breaking changes:

  • for 1. it installs pytorch==2.7.0 in github workflows when vLLM:main branch is used
  • for 2: it infers the task through model_config.supported_tasks rather than model_config.task now

@github-actions
Copy link

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

@sducouedic sducouedic changed the title install pytorch 2.7.0 or 2.5.1 depending on upstream branch [Fix] Tests breaking with vLLM:Main Jul 14, 2025
@sducouedic sducouedic marked this pull request as ready for review July 14, 2025 18:31
Copy link
Collaborator

@joerunde joerunde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Looks like the last failure is due to cleanup not happening properly after running with tensor parallel. We can go ahead and merge this and continue to investigate that failure.

Once we have a base image up including a 2.7.1 version of torch we can standardize on that as well

@joerunde joerunde merged commit 027acd9 into main Jul 14, 2025
38 of 42 checks passed
@joerunde joerunde deleted the torch_2_7_for_upstream_main_tests branch July 14, 2025 20:17
Copy link
Collaborator

@maxdebayser maxdebayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix tests issue with upstream main

4 participants