Skip to content

Conversation

FuxMak
Copy link

@FuxMak FuxMak commented Jul 7, 2025

Resolves #192 and #195.

I'm not an expert in Python, but the function was required for me to launch a vllm-worker with a Qwen3-30B-A3B and more than the 32k context window, as described in the runpod.io blogpost.

With the following environment variable and the changes, the container was able to run and pass the settings successfully to vllm. The function parse_json_env_var in my PR is not only used in the ROPE_SCALING var, but already all vars of type dict, as they will face the same issue.

ROPE_SCALING: {"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to set dict environment variables due to missing JSON parsing

1 participant