[Doc]: Does vllm CPU backend support Intel AMX?

[awangzy]

📚 The doc issue

I see pr #4971 has integrated some optimizations into vllm CPU backend, and I want to know whether vllm CPU supports intel AMX, and how to use it.
Thank you.

Suggest a potential alternative/fix

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[awangzy]

using intel-pcm tool, seems that amx is used by CPU backend

[RW6574]

is it using AMX-INT8 ? of only fp16 ?

[awangzy]

is it using AMX-INT8 ? of only fp16 ?

try model:https://modelscope.cn/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
dtype = bf16

[d4munche3z]

Also have the same question, that PR looks to note avx516 but would AMX be implemented also?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[MaxAmende]

Also have the same question, that PR looks to note avx516 but would AMX be implemented also?

Hi, did you figure out whether AVX512 (+vnni) or really AMX is used?

[phalani-paladugu]

AMX/AVX512 is used via OneDNN - This ?

[naveline67]

actually did test on sapphire rappids, it WORKS and uses AMX. either way try compile it from source; or use the CPU docker image.

did test on W7-2595X. Also not so much avx/tile offset donwclocking .

I did try the hard way to compile from source, but in fact i ended doing the same commands than the CPU docker image.

edit: using intel AMX INT8 with A8W8 (int8) models