Skip to content

Refactor LinearActQuantizedTensor #542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 26, 2024
Merged

Refactor LinearActQuantizedTensor #542

merged 1 commit into from
Jul 26, 2024

Conversation

jerryzh168
Copy link
Contributor

@jerryzh168 jerryzh168 commented Jul 25, 2024 •
edited
Loading

Summary:

  • rename to LinearActivationQuantizedTensor
  • using implements util to implement torch function and torch dispatch overwrites
  • refactored tensor subclass dispatch related utility functions: _implements and added two dispatch helpers: _dispatch__torch_function__ and _dispatch__torch_dispatch__ that saves more lines of code from users

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 25, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/542

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 097a61e with merge base c9f79be (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 25, 2024
jcaip
jcaip approved these changes Jul 25, 2024
View reviewed changes
@jerryzh168 jerryzh168 force-pushed the refactor-linear-act branch 4 times, most recently from 5007567 to 5f300dd Compare July 26, 2024 01:25
@jerryzh168
Copy link
Contributor Author

jerryzh168 commented Jul 26, 2024 •
edited
Loading

@jcaip please take a look again, the scope increased a bit to include a refactor for torch_function/torch_dispatch related utils functions

also @msaroufim @gau-nernst please take a look as well

@jerryzh168 jerryzh168 requested a review from gau-nernst July 26, 2024 01:29
gau-nernst
gau-nernst approved these changes Jul 26, 2024
View reviewed changes
Copy link
Collaborator

@gau-nernst gau-nernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new _dispatch__torch_function__ and _dispatch__torch_dispatch__ are very nice, reduce boilerplate. Apart from also making the changes to torchao/prototype/quant_llm/quant_llm.py, for the codes that I wrote, as long as the tests pass, they are good for me.

@jerryzh168 jerryzh168 force-pushed the refactor-linear-act branch from 5f300dd to 2ba9bfc Compare July 26, 2024 04:10
msaroufim
msaroufim approved these changes Jul 26, 2024
View reviewed changes
Copy link
Member

@msaroufim msaroufim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice clean change, just double checking this won't cause any BC issues on tune side for their nf4 recipe?

@jerryzh168 jerryzh168 force-pushed the refactor-linear-act branch 2 times, most recently from 70e760e to 1f849c2 Compare July 26, 2024 06:14
@jerryzh168
Refactor LinearActQuantizedTensor
097a61e 
Summary:
* rename to LinearActivationQuantizedTensor
* using `implements` util to implement torch function and torch dispatch overwrites

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:
@jerryzh168 jerryzh168 force-pushed the refactor-linear-act branch from 1f849c2 to 097a61e Compare July 26, 2024 16:24
@jerryzh168
Copy link
Contributor Author

jerryzh168 commented Jul 26, 2024 •
edited
Loading

nice clean change, just double checking this won't cause any BC issues on tune side for their nf4 recipe?

oh nf4 is not using these utils yet, I can do it in a separate PR

@jerryzh168 jerryzh168 merged commit afde175 into pytorch:main Jul 26, 2024
13 checks passed
@jerryzh168 jerryzh168 deleted the refactor-linear-act branch July 26, 2024 17:10
jerryzh168 added a commit to jerryzh168/ao that referenced this pull request Jul 26, 2024
@jerryzh168
Refactor NF4Tensor to use dispatch utils
f953281 
Summary:
As a follow up of pytorch#542, we can simplify the code
of nf4tensor by using the dispatch utils as well.

Test Plan:
python test/dtypes/test_nf4.py

Reviewers:

Subscribers:

Tasks:

Tags:
@jerryzh168 jerryzh168 mentioned this pull request Jul 26, 2024
Draft
jerryzh168 added a commit to jerryzh168/ao that referenced this pull request Jul 26, 2024
@jerryzh168
Refactor NF4Tensor to use dispatch utils
452901a 
Summary:
As a follow up of pytorch#542, we can simplify the code
of nf4tensor by using the dispatch utils as well.

Test Plan:
python test/dtypes/test_nf4.py

Reviewers:

Subscribers:

Tasks:

Tags:
Hanxian97 pushed a commit that referenced this pull request Jul 29, 2024
@jerryzh168 @Hanxian97
Refactor LinearActQuantizedTensor (#542)
83dc7cf 
Summary:
* rename to LinearActivationQuantizedTensor
* using `implements` util to implement torch function and torch dispatch overwrites

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:
dbyoung18 pushed a commit to dbyoung18/ao that referenced this pull request Jul 31, 2024
@jerryzh168
Refactor LinearActQuantizedTensor (pytorch#542)
86c399f 
Summary:
* rename to LinearActivationQuantizedTensor
* using `implements` util to implement torch function and torch dispatch overwrites

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:
@gau-nernst gau-nernst mentioned this pull request Aug 5, 2024
Merged
@jerryzh168 jerryzh168 mentioned this pull request Aug 5, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msaroufim msaroufim msaroufim approved these changes

@jcaip jcaip jcaip approved these changes

@gau-nernst gau-nernst gau-nernst approved these changes

@andrewor14 andrewor14 Awaiting requested review from andrewor14

@HDCharles HDCharles Awaiting requested review from HDCharles

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@jerryzh168 @msaroufim @jcaip @gau-nernst @facebook-github-bot