Add Ascend NPU as a new backend

[hipudding]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Feature Description

Add Ascend NPU as a new backend.

Motivation

Ascend is a full-stack AI computing infrastructure for industry applications and services based on Huawei Ascend processors and software. For more information about Ascend, see Ascend Community.

CANN (Compute Architecture of Neural Networks), developped by Huawei, is a heterogeneous computing architecture for AI.

Pytorch has officially announced support for Ascend NPU (through key PrivateUse1), please see the PrivateUse1 tutorial here.

Provide new backend support for llama.cpp, allowing users who are using Ascend NPU to inference model with llama.cpp.

Possible Implementation

Currently, the community has provided a convenient backend access mechanism. Ascend NPU is a CUDA-LIKE device, and I plan to reference CUDA's implementation to complete the Ascend NPU backend.

Due to the large workload, I plan to complete this feature in multiple stages. First, I will focus on compiling, backend registration, and device runtime functionalities. Additionally, I will add a new test file to validate backend registration, memory allocation, tensor operations, and other functionalities.

Next, I will proceed to implement tensor operators and validate them.

Afterward, do performance implementation, including split tensor support.

See also: very first commit #6035.

[hipudding]

@ggerganov Thanks for this awesome project, I'm looking forward to your suggestions.

[ggerganov]

Interesting work! Looks like a good start. Follow @slaren's advice for any insights with the implementation

[hipudding]

Interesting work! Looks like a good start. Follow @slaren's advice for any insights with the implementation

Thanks. I will.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.