Skip to content

Commit 516d88e

Browse files
authored
readme : add GPT4All instructions (close #588)
1 parent 53635c0 commit 516d88e

File tree

1 file changed

+18
-3
lines changed

1 file changed

+18
-3
lines changed

README.md

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,7 @@ Inference of [LLaMA](https://arxiv.org/abs/2302.13971) model in pure C/C++
1010
**Hot topics:**
1111

1212
- [Roadmap (short-term)](https://github.com/ggerganov/llama.cpp/discussions/457)
13-
- New C-style API is now available: https://github.com/ggerganov/llama.cpp/pull/370
14-
- Cache input prompts for faster initialization: https://github.com/ggerganov/llama.cpp/issues/64
15-
- Create a `llama.cpp` logo: https://github.com/ggerganov/llama.cpp/issues/105
13+
- Support for [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all)
1614

1715
## Description
1816

@@ -37,6 +35,12 @@ Supported platforms:
3735
- [X] Windows (via CMake)
3836
- [X] Docker
3937

38+
Supported models:
39+
40+
- [X] LLaMA
41+
- [X] [Alpaca](https://github.com/ggerganov/llama.cpp#instruction-mode-with-alpaca)
42+
- [X] [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all)
43+
4044
---
4145

4246
Here is a typical run using LLaMA-7B:
@@ -222,6 +226,17 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach.
222226
>
223227
```
224228

229+
### Using [GPT4All](https://github.com/nomic-ai/gpt4all)
230+
231+
- Obtain the `gpt4all-lora-quantized.bin` model
232+
- It is distributed in the old `ggml` format which is not obsoleted. So you have to convert it to the new format using [./convert-gpt4all-to-ggml.py](./convert-gpt4all-to-ggml.py):
233+
234+
```bash
235+
python3 convert-gpt4all-to-ggml.py models/gpt4all-7B/gpt4all-lora-quantized.bin ./models/tokenizer.model
236+
```
237+
238+
- You can now use the newly generated `gpt4all-lora-quantized.bin` model in exactly the same way as all other models. The original model is stored in the same folder with a suffix `.orig`
239+
225240
### Obtaining and verifying the Facebook LLaMA original model and Stanford Alpaca model data
226241

227242
- **Under no circumstances share IPFS, magnet links, or any other links to model downloads anywhere in this respository, including in issues, discussions or pull requests. They will be immediately deleted.**

0 commit comments

Comments
 (0)