Skip to content

[Quantizers] add is_compileable property to quantizers. #11736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 19, 2025
Merged

[Quantizers] add is_compileable property to quantizers. #11736

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
edb782c
add is_compileable property to quantizers.
sayakpaul Jun 18, 2025
58e19d1
Merge branch 'main' into is-compileable-prop-quant
sayakpaul Jun 18, 2025
507107e
Merge branch 'main' into is-compileable-prop-quant
sayakpaul Jun 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/diffusers/quantizers/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,3 +227,8 @@ def is_serializable(self): ...
@property
@abstractmethod
def is_trainable(self): ...

@property
def is_compileable(self) -> bool:
"""Flag indicating whether the quantized model can be compiled"""
return False
4 changes: 4 additions & 0 deletions src/diffusers/quantizers/bitsandbytes/bnb_quantizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -564,6 +564,10 @@ def is_trainable(self) -> bool:
# Because we're mandating `bitsandbytes` 0.43.3.
return True

@property
def is_compileable(self) -> bool:
return True

Comment on lines +567 to +570
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add check for bnb version

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not sure about version-guarding since compilation was supported even before the latest release. It's the "no graph-breaks" that is the catch.

def _dequantize(self, model):
from .utils import dequantize_and_replace

Expand Down
4 changes: 4 additions & 0 deletions src/diffusers/quantizers/gguf/gguf_quantizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,10 @@ def is_serializable(self):
def is_trainable(self) -> bool:
return False

@property
def is_compileable(self) -> bool:
return True

def _dequantize(self, model):
is_model_on_cpu = model.device.type == "cpu"
if is_model_on_cpu:
Expand Down
4 changes: 4 additions & 0 deletions src/diffusers/quantizers/quanto/quanto_quantizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,3 +175,7 @@ def is_trainable(self):
@property
def is_serializable(self):
return True

@property
def is_compileable(self) -> bool:
return True
4 changes: 4 additions & 0 deletions src/diffusers/quantizers/torchao/torchao_quantizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,3 +335,7 @@ def is_serializable(self, safe_serialization=None):
@property
def is_trainable(self):
return self.quantization_config.quant_type.startswith("int8")

@property
def is_compileable(self) -> bool:
return True