You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Machine Learning Compilation for Large Language Models (MLC LLM) is a high-performance universal deployment solution that allows native deployment of any large language models with native APIs with compiler acceleration. The mission of this project is to enable everyone to develop, optimize and deploy AI models natively on everyone's devices with ML compilation techniques.
7
+
**M**achine **L**earning **C**ompilation for **L**arge **L**anguage **M**odels (MLC LLM) is a high-performance universal deployment solution that allows native deployment of any large language models with native APIs with compiler acceleration. The mission of this project is to enable everyone to develop, optimize and deploy AI models natively on everyone's devices with ML compilation techniques.
8
8
9
9
**Universal deployment.** MLC LLM supports the following platforms and hardware:
10
10
11
11
<tablestyle="width:100%">
12
12
<thead>
13
13
<tr>
14
-
<th style="width:16%"> </th>
15
-
<th style="width:21%">AMD GPU</th>
16
-
<th style="width:21%">NVIDIA GPU</th>
17
-
<th style="width:21%">Apple M1/M2 GPU</th>
18
-
<th style="width:21%">Intel GPU</th>
14
+
<th style="width:15%"> </th>
15
+
<th style="width:20%">AMD GPU</th>
16
+
<th style="width:20%">NVIDIA GPU</th>
17
+
<th style="width:20%">Apple GPU</th>
18
+
<th style="width:24%">Intel GPU</th>
19
19
</tr>
20
20
</thead>
21
21
<tbody>
@@ -28,21 +28,18 @@ Machine Learning Compilation for Large Language Models (MLC LLM) is a high-perfo
28
28
</tr>
29
29
<tr>
30
30
<td>macOS</td>
31
-
<td>✅ Metal</td>
31
+
<td>✅ Metal (dGPU)</td>
32
32
<td>N/A</td>
33
33
<td>✅ Metal</td>
34
-
<td>✅ Metal</td>
34
+
<td>✅ Metal (iGPU)</td>
35
35
</tr>
36
36
<tr>
37
37
<td>Web Browser</td>
38
-
<td>✅ WebGPU</td>
39
-
<td>✅ WebGPU</td>
40
-
<td>✅ WebGPU</td>
41
-
<td>✅ WebGPU</td>
38
+
<td colspan=4>✅ WebGPU and WASM </td>
42
39
</tr>
43
40
<tr>
44
41
<td>iOS / iPadOS</td>
45
-
<td colspan=4>✅ Metal on Apple M1/M2 GPU</td>
42
+
<td colspan=4>✅ Metal on Apple A-series GPU</td>
46
43
</tr>
47
44
<tr>
48
45
<td>Android</td>
@@ -69,16 +66,16 @@ Machine Learning Compilation for Large Language Models (MLC LLM) is a high-perfo
69
66
70
67
Please visit our [documentation](https://llm.mlc.ai/docs/index.html#getting-started) for detailed instructions.
71
68
72
-
## Prebuilt model support
69
+
## Model Support
73
70
74
71
MLC LLM supports a wide range of model architectures and variants. We have the following prebuilts which you can
75
72
use off-the-shelf. Visit [Prebuilt Models](https://llm.mlc.ai/docs/prebuilt_models.html) to see the full list, and [Compile Models via MLC](https://llm.mlc.ai/docs/compilation/compile_models.html) to see how to use models not on this list.
76
73
77
74
<tablestyle="width:100%">
78
75
<thead>
79
76
<tr>
80
-
<th style="width:40%">Supported Model Architecture</th>
81
-
<th style="width:60%">Model Variants with Prebuilts</th>
77
+
<th style="width:40%">Architecture</th>
78
+
<th style="width:60%">Prebuilt Model Variants</th>
0 commit comments