Skip to content

Commit a551933

Browse files
authored
cann : add Ascend NPU instructions (#2410)
1 parent 5caa192 commit a551933

File tree

1 file changed

+34
-0
lines changed

1 file changed

+34
-0
lines changed

README.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ High-performance inference of [OpenAI's Whisper](https://github.com/openai/whisp
2121
- Support for CPU-only inference
2222
- [Efficient GPU support for NVIDIA](https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas)
2323
- [OpenVINO Support](https://github.com/ggerganov/whisper.cpp#openvino-support)
24+
- [Ascend NPU Support](https://github.com/ggerganov/whisper.cpp#ascend-npu-support)
2425
- [C-style API](https://github.com/ggerganov/whisper.cpp/blob/master/include/whisper.h)
2526

2627
Supported platforms:
@@ -448,6 +449,39 @@ cmake -DWHISPER_MKL=ON ..
448449
WHISPER_MKL=1 make -j
449450
```
450451

452+
## Ascend NPU support
453+
454+
Ascend NPU provides inference acceleration via [`CANN`](https://www.hiascend.com/en/software/cann) and AI cores.
455+
456+
First, check if your Ascend NPU device is supported:
457+
458+
**Verified devices**
459+
| Ascend NPU | Status |
460+
|:-----------------------------:|:-------:|
461+
| Atlas 300T A2 | Support |
462+
463+
Then, make sure you have installed [`CANN toolkit`](https://www.hiascend.com/en/software/cann/community) . The lasted version of CANN is recommanded.
464+
465+
Now build `whisper.cpp` with CANN support:
466+
467+
```
468+
mkdir build
469+
cd build
470+
cmake .. -D GGML_CANN=on
471+
make -j
472+
```
473+
474+
Run the inference examples as usual, for example:
475+
476+
```
477+
./build/bin/main -f samples/jfk.wav -m models/ggml-base.en.bin -t 8
478+
```
479+
480+
*Notes:*
481+
482+
- If you have trouble with Ascend NPU device, please create a issue with **[CANN]** prefix/tag.
483+
- If you run successfully with your Ascend NPU device, please help update the table `Verified devices`.
484+
451485
## Docker
452486

453487
### Prerequisites

0 commit comments

Comments
 (0)