Skip to content

Support fp16 model to weight-only quantization for PyTorch framework #1387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Nov 16, 2023
Merged

Support fp16 model to weight-only quantization for PyTorch framework #1387

merged 3 commits into from
Nov 16, 2023

Conversation

PenghuiCheng
Copy link
Contributor

Type of Change

feature
No API changed

Description

Support fp16 model to weight-only quantization for PyTorch framework.

Expected Behavior & Potential Risk

Quantization fp16 model successfully.

How has this PR been tested?

local tested

@PenghuiCheng PenghuiCheng requested a review from xin3he November 15, 2023 06:57
@PenghuiCheng
Add UT
91d60c9 
Signed-off-by: Cheng, Penghui <[email protected]>
xin3he
xin3he reviewed Nov 15, 2023
View reviewed changes
Copy link
Contributor

@xin3he xin3he left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think It's better to move the model.float() into the for loopfor name, m in model.named_modules():

@PenghuiCheng
update code
8399d64 
Signed-off-by: Cheng, Penghui <[email protected]>
@PenghuiCheng
Copy link
Contributor Author

I think It's better to move the model.float() into the for loopfor name, m in model.named_modules():

yes, changed

@chensuyue chensuyue merged commit d5cb567 into master Nov 16, 2023
@chensuyue chensuyue deleted the penghuic/support_fp16_for_woq branch November 16, 2023 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xin3he xin3he xin3he approved these changes

@chensuyue chensuyue chensuyue approved these changes

@changwangss changwangss Awaiting requested review from changwangss

@kevinintel kevinintel Awaiting requested review from kevinintel

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@PenghuiCheng @chensuyue @xin3he