[Feature]: Realtime ASR

[gaardhus]

🚀 The feature, motivation and pitch

I need super short latency on a transcription service, so having realtime ASR implemented in vLLM would be awesome.

See i.e.
https://platform.openai.com/docs/guides/speech-to-text?lang=curl#streaming-the-transcription-of-an-ongoing-audio-recording
https://github.com/QuentinFuxa/WhisperLiveKit

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

We need to support streaming inputs in order to enable this: #25066