Skip to content

Fix MLX-VLM snippet and prio hf_xet for MLX models #1463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 20, 2025

Conversation

Vaibhavs10
Copy link
Member

Simple PR to put pip install instructions in comments
and hf_transfer -> hf_xet for MLX (since the models have moved over to xet backend.

re: MLX VLM - currently there is a broken comment at the top here: https://huggingface.co/mlx-community/gemma-3-27b-it-qat-4bit?library=mlx

@@ -1355,15 +1355,14 @@ model = SwarmFormerModel.from_pretrained("${model.id}")

const mlx_unknown = (model: ModelData): string[] => [
`# Download the model from the Hub
pip install huggingface_hub hf_transfer
pip install huggingface_hub[hf_xet]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc: @jsulz - should we put any other env variables here? IIRC no, but wondering if you'd recommend any!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put this here: 3c5ab82


export HF_HUB_ENABLE_HF_TRANSFER=1
export HF_XET_HIGH_PERFORMANCE=1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest we remove this to make everything simpler. In my experience it works fast enough without the flag.

huggingface-cli download --local-dir ${nameWithoutNamespace(model.id)} ${model.id}`,
];

const mlxlm = (model: ModelData): string[] => [
`# Make sure mlx-lm is installed
pip install --upgrade mlx-lm
# pip install --upgrade mlx-lm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah good call, no longer a cli

Copy link
Member Author

@Vaibhavs10 Vaibhavs10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

failing tests look unrelated

@Vaibhavs10 Vaibhavs10 merged commit e55e687 into main May 20, 2025
4 of 5 checks passed
@Vaibhavs10 Vaibhavs10 deleted the vb/fix-mlx-vlm-snippet branch May 20, 2025 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants