Support whisper-large-{v2, v3, v3-turbo} #15680
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15680
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 7 Pending, 3 Unrelated FailuresAs of commit 178ec40 with merge base 3dbc15b ( NEW FAILURE - The following job has failed:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
Gasoonjia
requested review from
larryliu0820,
manuelcandales and
mergennachin
| *) | ||
| echo "Error: Unsupported model '$HF_MODEL'" | ||
| echo "Supported models: mistralai/Voxtral-Mini-3B-2507, openai/whisper-small, google/gemma-3-4b-it" | ||
| echo "Supported models: mistralai/Voxtral-Mini-3B-2507, openai/whisper-small, openai/whisper-large-v2, google/gemma-3-4b-it" |
| Supported models: | ||
| - mistralai/Voxtral-Mini-3B-2507 | ||
| - openai/whisper-small | ||
| - openai/whisper-large-v2 |
There was a problem hiding this comment.
I think we should add large-v3-turbo in CI.
examples/models/whisper/main.cpp
Outdated
| DEFINE_string( | ||
| model_name, | ||
| "base", | ||
| "Whisper model name (base, small, medium, large, large-v2, large-v3, turbo)."); |
extension/asr/runner/runner.cpp
Outdated
| static_cast<float>( | ||
| encoder_output_tensor | ||
| .mutable_data_ptr<::executorch::aten::BFloat16>()[0])); | ||
| .mutable_data_ptr<float>()[0])); |
There was a problem hiding this comment.
this is where the encoder input == 0 came from.
| PREPROCESSOR_OUTPUT="voxtral_preprocessor.pte" | ||
| ;; | ||
| openai/whisper-small) | ||
| openai/whisper-*) |
There was a problem hiding this comment.
PREPROCESSOR_FEATURE_SIZE changes depending on the whisper model
There was a problem hiding this comment.
thx for pointed out! Fixeed it
This PR supports whisper-large-{v2, v3, v3-turbo} on et-cuda backend, and update readme and ci.