Move cutlass_int4_packed_layout to prototype #3277
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Moved cutlass_int4_packed_layout.py from torchao/dtypes/uintx/ to torchao/prototype/dtypes/uintx/ - Created torchao/prototype/dtypes/__init__.py and uintx/__init__.py to export CutlassInt4PackedLayout - Replaced old file with backward compatibility stub that imports from new location - Added deprecation warning for old import path (to be removed in v0.16.0) - Updated torchao/dtypes/__init__.py to re-export from prototype for backward compatibility - Updated internal imports in affine_quantized_tensor_ops.py to use new prototype location - Removed CutlassInt4PackedLayout from torchao/dtypes/uintx/__init__.py to avoid circular imports - Updated documentation to move CutlassInt4PackedLayout to Prototype section All import paths work: - New: from torchao.prototype.dtypes import CutlassInt4PackedLayout - Backward compat: from torchao.dtypes import CutlassInt4PackedLayout - Deprecated: from torchao.dtypes.uintx.cutlass_int4_packed_layout import CutlassInt4PackedLayout
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3277
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 42acafa with merge base 315e9b4 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
17 tasks
Removed unused import for CutlassInt4PackedLayout.
Removed __all__ exports from cutlass_int4_packed_layout.py
jainapurva
force-pushed
the
move_cutlass_int4_packed_layout
branch
from
6c03ed4 to
8c18d4d
Compare
jainapurva
added
the
topic: bc-breaking
jerryzh168
reviewed
Nov 4, 2025
docs/source/api_ref_dtypes.rst
Outdated
| FloatxTensor | ||
| FloatxTensorCoreLayout | ||
| MarlinSparseLayout | ||
| BlockSparseLayout |
There was a problem hiding this comment.
looks like it's both? maybe try to split would be better, or update the summary to say it's two things
There was a problem hiding this comment.
It's in two separate PRs:
Pr for BlockSparseLayout: #3276
This one is for CutlassInt4PackedLayout
jerryzh168
reviewed
Nov 4, 2025
| from torchao.dtypes.utils import ( | ||
| Layout, | ||
| PlainLayout, | ||
| warnings.warn( |
There was a problem hiding this comment.
add a test for warning as well? example
jerryzh168
reviewed
Nov 4, 2025
| if bias is not None: | ||
| y += bias | ||
| return y | ||
| from torchao.prototype.dtypes.uintx.block_sparse_layout import ( |
There was a problem hiding this comment.
since this keeps BC I think we should remove the bc-breaking label
…utlass_int4_packed_layout
jerryzh168
approved these changes
Nov 4, 2025
vkuzo
reviewed
Nov 5, 2025
| ) | ||
| from torchao.dtypes.uintx.plain_layout import ( | ||
| _aqt_is_int8, | ||
| warnings.warn( |
There was a problem hiding this comment.
also, I'd remove torchao v0.16.0 and just say "in a future release of torchao", just in case the work gets delayed
jainapurva
force-pushed
the
move_block_sparsity
branch
from
8ebbb9c to
9cce063
Compare
jainapurva
force-pushed
the
move_block_sparsity
branch
from
9cce063 to
78f5e4c
Compare
…utlass_int4_packed_layout
jainapurva
force-pushed
the
move_block_sparsity
branch
from
78f5e4c to
1a75689
Compare
…utlass_int4_packed_layout
jainapurva
force-pushed
the
move_block_sparsity
branch
from
1a75689 to
9bff162
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request moves
CutlassInt4PackedLayoutand its implementation fromtorchao/dtypes/uintx/cutlass_int4_packed_layout.pytotorchao/prototype/dtypes/uintx/cutlass_int4_packed_layout.py, and exposed it viatorchao.prototype.dtypeswhile maintaining BC attorchao.prototype.dtypes.uintxwith a deprecation warning for torchao v0.16.0.Updates API documentation to reflect the changes
Reference Issue: #2752