Can I use whisper .cpp on iOS and Android to directly convert speech to Chinese?

[xclidongbo]

Can I use whisper .cpp on iOS and Android to directly convert speech to Chinese?

[ChaosTong]

sure, just do not select .en model. you can use auto to detect language or just select specific language like this:

./main -osrt -m ./models/ggml-base.bin -f samples/test1.wav -l zh

[xclidongbo]

@ChaosTong
The mobile model is too large, it is not convenient to cram into the mobile terminal, and the recognition accuracy of Tiny is not high. What are the mature solutions?

[ChaosTong]

you can make tiny as default model, let user handle which model their device can handle, or you detect user's mobile model help their set the biggest model they can use.

[xclidongbo]

Tested using a demo of React Native. It was found that the SDK on the iOS side, under tiny's model, the converted content is inaccurate, and with 100k war audio, the conversion time takes 40-50s.

[aehlke]

Any luck running models >tiny on iOS?

[Mickeyha]

Hi, for whisper.Android, I just put "for-tests-ggml-base.bin" under Whisper/whisper.cpp/examples/whisper.android/app/src/main/assets. And the icon can be pressed -> which means the model have been loaded correctly. But I can not get any outputs with both "recording" or "transcribe sample". The Done(xxx ms) string is with nothing after.

[phamkhactu]

Hi @xclidongbo @ChaosTong

I saw discussion which is a discussion about improving acc. Have you ever used as streaming ?