Quant fallback to 8w per token + other quant improvements for multimodal

[jackzhxng]

Big quantization improvements for Gemma3 4B vision (7.4 GB -> 3.0 GB)

• Quantize encoder with 8da4w group size 32 where possible, else use 8da8w per token (this applies to the fc2 layers)
• Quantize LM head

optimum-cli export executorch
    --model google/gemma-3-4b-it
    --task "multimodal-text-to-text"
    --max_seq_len 1024
    --recipe "xnnpack"
    --use_custom_sdpa
    --use_custom_kv_cache
    --qlinear 8da4w
    --qlinear_group_size 32
    --qlinear_encoder 8da4w,8da8w
    --qlinear_encoder_group_size 32
    --qembedding 8w
    --output_dir="gemma3_vision"

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[metascroy]

Why is groupsixe a multiple of 2? Shouldn't it be a multiple of 32?

[larryliu0820]

Can you guard this by whether eager_model has lm_head?

[jackzhxng]

Sure, curious though is there a model without lm_head?

[larryliu0820]

Yeah voxtral doesn't have lm_head