-
Notifications
You must be signed in to change notification settings - Fork 408
Fix MLX-VLM snippet and prio hf_xet for MLX models #1463
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@@ -1355,15 +1355,14 @@ model = SwarmFormerModel.from_pretrained("${model.id}") | |||
|
|||
const mlx_unknown = (model: ModelData): string[] => [ | |||
`# Download the model from the Hub | |||
pip install huggingface_hub hf_transfer | |||
pip install huggingface_hub[hf_xet] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc: @jsulz - should we put any other env variables here? IIRC no, but wondering if you'd recommend any!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
put this here: 3c5ab82
|
||
export HF_HUB_ENABLE_HF_TRANSFER=1 | ||
export HF_XET_HIGH_PERFORMANCE=1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd suggest we remove this to make everything simpler. In my experience it works fast enough without the flag.
huggingface-cli download --local-dir ${nameWithoutNamespace(model.id)} ${model.id}`, | ||
]; | ||
|
||
const mlxlm = (model: ModelData): string[] => [ | ||
`# Make sure mlx-lm is installed | ||
pip install --upgrade mlx-lm | ||
# pip install --upgrade mlx-lm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah good call, no longer a cli
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
failing tests look unrelated
Simple PR to put pip install instructions in comments
and
hf_transfer
->hf_xet
for MLX (since the models have moved over to xet backend.re: MLX VLM - currently there is a broken comment at the top here: https://huggingface.co/mlx-community/gemma-3-27b-it-qat-4bit?library=mlx