Skip to content

Conversation

@ahao-anyscale
Copy link
Contributor

@ahao-anyscale ahao-anyscale commented Sep 17, 2025

Purpose

When pulling files for model and tokenizer in maybe_pull_model_tokenizer_for_runai, *.pth files are not excluded from being pulled. Some huggingface repos like https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/tree/main/original contain .pth files, which adds a significant amount of unnecessary latency to startup times.

Test Plan

I timed the ModelConfig creation time and ensured that the startup latency went down.

Test Result

N/A


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@ahao-anyscale ahao-anyscale marked this pull request as ready for review September 17, 2025 18:37
@kouroshHakha kouroshHakha added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 17, 2025
@simon-mo simon-mo enabled auto-merge (squash) September 17, 2025 20:11
@simon-mo simon-mo merged commit f20c3b0 into vllm-project:main Sep 17, 2025
54 of 56 checks passed
debroy-rh pushed a commit to debroy-rh/vllm that referenced this pull request Sep 19, 2025
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants