[Feature] Support Zero-point Decompression

[dsikka]

Is your feature request related to a problem? Please describe.

• For the packed compressor, when applying asymmetric quantization, the zero-point is also packed. There is however, no support for decompressing / unpacking the zero-point. We need to update the compressor to support zero-point unpaacking / decompression

Describe the solution you'd like

• Support zero-point decompression if running asymmetric quantization using the packed-compressor
• Quantized compressors only support weight-decompression, as seen here: https://github.com/neuralmagic/compressed-tensors/blob/c69c8bbf6ba93ecb8d86b8837e86b7abda8568af/src/compressed_tensors/compressors/quantized_compressors/base.py#L211
• Add tests to validate this case - you should be able to load this model through AutoModelForCasualLM and run generations without issue

[aladerran]

I will be working on it but feel free to ping / collaborate with me for other contributors

[dsikka]

Hi @aladerran - are you working on this issue?

[aladerran]

@dsikka I'm not actively working on this at the moment as I lack the computing resources T.T

[Etelis]

If you're unavailable @aladerran I will take this one.
Thanks :)

[dsikka]

@Etelis - Thanks! Will assign the task to you

[Etelis]

@dsikka, Quick clarification on scope: do we need changes in llm-compressor itself to support zero-point decompression for the packed/asymmetric path, or is this entirely an upstream change in compressed_tensors?
From what I see, llm-compressor delegates both packing and decompression to compressed_tensors via ModelCompressor (e.g., compressed_tensors_utils.get_model_compressor, compressor.decompress_model(model)), so it seems the fix should live in compressed_tensors (zero-point unpack + dequant), followed by a dependency bump here.

[dsikka]

@Etelis yup, that's exactly what we would need to update.
Thank you for taking this on!