[bug] tgi-1.1.0 - Please install EETQ from https://github.com/NetEase-FuXi/EETQ #3377
Description
Checklist
- I've prepended issue tag with type of change: [bug]
- (If applicable) I've attached the script to reproduce the bug
- (If applicable) I've documented below the DLC image/dockerfile this relates to
- (If applicable) I've documented below the tests I've run on the DLC image
- I'm using an existing DLC image listed here: https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html
- I've built my own container based off DLC (and I've attached the code used to build my own image)
Concise Description:
When deploying a model with EETQ HF_MODEL_QUANTIZE, it results in the error "ImportError: Please install EETQ from https://github.com/NetEase-FuXi/EETQ".
DLC image/dockerfile:
- v1.0-hf-tgi-1.1.0-pt-2.0.1-inf-gpu-py39
- 763104351884.dkr.ecr.us-east-2.amazonaws.com/huggingface-pytorch-tgi-inference:2.0.1-tgi1.1.0-gpu-py39-cu118-ubuntu20.04-v1.0
Current behavior:
Returns with error ImportError: Please install EETQ from https://github.com/NetEase-FuXi/EETQ
Expected behavior:
The model should deploy and run without any issues.
Additional context: