[Bug]: [V1] wrong output when using kv cache fp8

[sleepwalker2017]

Your current environment

KV fp8 is ok when disabling V1 Engine, But when using V1 engine, the output is totally wrong.

vllm version 0.7.1, GPU H100

port = 10011
model = "/data/models/qwen2.5_72b-FP8"

#cmd = f"VLLM_USE_V1=1 python3 -m vllm.entrypoints.openai.api_server \
cmd = f"VLLM_USE_V1=1 python3 -m vllm.entrypoints.openai.api_server \
        --port {port} \
        --model {model} \
        --dtype auto \
        -tp 2 \
        --max-model-len 8192 \
        --max-num-seqs 512 \
        --gpu-memory-utilization 0.9 \
        --enable-prefix-caching \
        --swap-space 16 \
        --disable-log-stats \
        --disable-log-requests \
        --trust-remote-code --kv-cache-dtype fp8"

🐛 Describe the bug

When disabling kv fp8, output length is 140. But when enabling it, output length is 1000+, and the text is unreadable.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Syst3m1cAn0maly]

I have an example output for this chat prompt :

{
"model": "llama-3-3-70b",
"messages": [
{
"role": "user",
"content": "hello"
}
],
"max_tokens": 20,"stream": "False",
"temperature": 0.7
}

V0 engine output with Llama 3.3 70b FP8 KV on vLLM v0.7.2:

{"id":"chatcmpl-8aecafcb7b5f41739a3e72fa0c9ea12c",
"object":"chat.completion",
"created":1739608319,
"model":"llama-3-3-70b"
,"choices":[{
        "index":0,
        "message":{
            "role":"assistant",
            "reasoning_content":null,
            "content":"Hello! It's nice to meet you. Is there something I can help you with or would you",
            "tool_calls":[]
        },
       "logprobs":null,
       "finish_reason":"length",
       "stop_reason":null
}],
"usage":{"prompt_tokens":36,"total_tokens":56,"completion_tokens":20,"prompt_tokens_details":null},"prompt_logprobs":null}

V1 engine output with Llama 3.3 70b FP8 KV on vLLM v0.7.2:

{"id":"chatcmpl-af445cbed5864504b596702c06f5fae9",
"object":"chat.completion",
"created":1739608624,
"model":"llama-3-3-70b",
,"choices":[{
        "index":0,
        "message":{
            "role":"assistant",
            "reasoning_content":null,
            "content":"Thus,QuestionMark Schmidt.Questiony A A .0ं‍� with c A players in",
            "tool_calls":[]
         },
         "logprobs":null,
         "finish_reason":"length",
         "stop_reason":null
}],
"usage":{"prompt_tokens":36,"total_tokens":56,"completion_tokens":20,"prompt_tokens_details":null},"prompt_logprobs":null}

[mgoin]

Thanks for the report. FP8 kv cache is not supported at all in V1 yet, so I am fixing the visibility here #14221