Skip to content

Commit 4003232

Browse files
WoosukKwonLeiWang1999
authored andcommitted
Revert "[Doc] Update supported_hardware.rst (vllm-project#7276)" (vllm-project#7467)
Signed-off-by: LeiWang1999 <[email protected]>
1 parent c37e446 commit 4003232

File tree

1 file changed

+13
-15
lines changed

1 file changed

+13
-15
lines changed

docs/source/quantization/supported_hardware.rst

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5,20 +5,18 @@ Supported Hardware for Quantization Kernels
55

66
The table below shows the compatibility of various quantization implementations with different hardware platforms in vLLM:
77

8-
===================== ====== ======= ======= ===== ====== ======= ========= ======= ============== ==========
9-
Implementation Volta Turing Ampere Ada Hopper AMD GPU Intel GPU x86 CPU AWS Inferentia Google TPU
10-
===================== ====== ======= ======= ===== ====== ======= ========= ======= ============== ==========
11-
AWQ ❌ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
12-
GPTQ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
13-
Marlin (GPTQ/AWQ/FP8) ❌ ❌ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
14-
INT8 (W8A8) ❌ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
15-
FP8 (W8A8) ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ❌
16-
AQLM ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
17-
bitsandbytes ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
18-
DeepSpeedFP ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
19-
GGUF ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
20-
SqueezeLLM ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
21-
===================== ====== ======= ======= ===== ====== ======= ========= ======= ============== ==========
8+
============== ====== ======= ======= ===== ====== ======= ========= ======= ============== ==========
9+
Implementation Volta Turing Ampere Ada Hopper AMD GPU Intel GPU x86 CPU AWS Inferentia Google TPU
10+
============== ====== ======= ======= ===== ====== ======= ========= ======= ============== ==========
11+
AQLM ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
12+
AWQ ❌ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
13+
DeepSpeedFP ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
14+
FP8 ❌ ❌ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
15+
Marlin ❌ ❌ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
16+
GPTQ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
17+
SqueezeLLM ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
18+
bitsandbytes ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
19+
============== ====== ======= ======= ===== ====== ======= ========= ======= ============== ==========
2220

2321
Notes:
2422
^^^^^^
@@ -29,4 +27,4 @@ Notes:
2927

3028
Please note that this compatibility chart may be subject to change as vLLM continues to evolve and expand its support for different hardware platforms and quantization methods.
3129

32-
For the most up-to-date information on hardware support and quantization methods, please check the `quantization directory <https://github.com/vllm-project/vllm/tree/main/vllm/model_executor/layers/quantization>`_ or consult with the vLLM development team.
30+
For the most up-to-date information on hardware support and quantization methods, please check the `quantization directory <https://github.com/vllm-project/vllm/tree/main/vllm/model_executor/layers/quantization>`_ or consult with the vLLM development team.

0 commit comments

Comments
 (0)