Add export_dtype parameter to convert_to_hf function
#2041
Conversation
idoh
requested review from
fegin,
tianyu-l,
wconstab and
wwwjn
as code owners
meta-cla
bot
added
the
CLA Signed
wwwjn
left a comment
wwwjn
left a comment
There was a problem hiding this comment.
Thanks for contributing, I have a basic question - When do we need low precision weights?
|
|
||
| @torch.inference_mode() | ||
| def convert_to_hf(input_dir, output_dir, model_name, model_flavor, hf_assets_path): | ||
| if model_name == "flux": |
There was a problem hiding this comment.
Why we are removing this line here? Could you provide some tests results (can be simple as a screenshot or terminal outputs) to show it still works for FLUX model?
There was a problem hiding this comment.
Good catch, I removed it because the import didn't make sense anymore as the flux folder was moved from experiments to the main models folders. So now it does not require "special" treatment.
There was a problem hiding this comment.
Regarding when to use low precision, it is more space efficient, especially when uploading checkpoints to cloud storage. I kept the default export fp32 but IMO making bf16 the default export type makes more sense as most models that are uploaded today are in bf16.
|
Thank you! Could you fix the |
idoh
force-pushed
the
hf-conversion-dtype
branch
from
8edfa3d to
9ce21b6
Compare
|
@wwwjn Thanks, I fixed it so now the linting should pass. |
3 participants
The current
convert_to_hf.pydoes not supportexport_dtype, which makes itfloat32by default. This PR adds support for export dtypes of["float16", "bfloat16", "float32"].