[model_free_ptq] Add pathway for day-zero weight quantization support

[kylesayrs]

Purpose

• Create a pathway which can quantize model weights without needing a model definition or the use of a calibration pipeline. Such a pathway provides fast and reliable support for models which:
  • Do not have a HF model definition yet
  • Have complications with sequential pipelines (very large vision towers, tracing failure, long calibration runtime)

Usage

model_free_ptq(
    model_stub="meta-llama/Llama-3.2-1B-Instruct",
    save_directory="Llama-3.2-1B-Instruct-FP8_block",
    scheme="FP8_BLOCK",
    ignore=["model.embed_tokens", "lm_head"],
    max_workers=15,
    device="cuda:0",
):

Testing

• Added test_model_free_ptq_matches_oneshot which tests that saved tensors and configs exactly match between model_free_ptq and oneshot entrypoints for the same arguments. This test takes about 10 seconds to run.

Future Extensions

• Mixed-precision quantization (multiple recipes/targets)
• Multi-GPU support (work is already parallelized by threads, but if GPU is the bottleneck we can split the work across GPUs)
• Multi-process support (is python processing is the bottleneck, we can replace multithreading with multiprocessing)

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[dsikka]

I think keeping this as a separate entrypoint makes sense given the point is speed of execution / not wanting it getting bogged down from other flows in llmcomp but it seems like the lifecycle / steps we're applying are very similar to the datafree pipeline?

[kylesayrs]

@dsikka Yes

[brian-dellabetta]

Great addition! One nit, and probably want to add an example or README to explain when one would want to use this over oneshot, but can also tackle that in a follow up as a good first issue.

[shanjiaz]

Great work!!