[Bug]: Deepseek-v3 performace on benchmark didn't match with paper

[jongjyh]

Your current environment

The output of `python collect_env.py`

Your output of `python collect_env.py` here

Model Input Dumps

No response

🐛 Describe the bug

Hi guys,

I used vllm to serve deepseek-v3 while I found the benchmark didn't reproduce the result on paper. Specifically, in my case deepseek-v3 got 82 on CEval comparing to 90 on paper.

Are there any details I missed? be willing to reveal any detail of my settings.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Wen1163204547]

遇到相同问题，搭个车
deepseek-ai/DeepSeek-V3#212

[SunflowerAries]

As far as I know, there exist some difference between vllm and DeepSeek v3's official inference code.

#12002

[Wen1163204547]

As far as I know, there exist some difference between vllm and DeepSeek v3's official inference code.

#12002

Thank you for your help. After the modification, it indeed aligns well. Many thanks!

[jongjyh]

Thanks!!