Skip to content

Commit c31bbe9

Browse files
authored
readme : add docs for chat-persistent.sh (#1568)
* readme : add docs for chat-persistent.sh * Update README.md
1 parent 1359b6a commit c31bbe9

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

README.md

+19
Original file line numberDiff line numberDiff line change
@@ -391,6 +391,25 @@ Note the use of `--color` to distinguish between user input and generated text.
391391
392392
![image](https://user-images.githubusercontent.com/1991296/224575029-2af3c7dc-5a65-4f64-a6bb-517a532aea38.png)
393393
394+
### Persistent Interaction
395+
396+
The prompt, user inputs, and model generations can be saved and resumed across calls to `./main` by leveraging `--prompt-cache` and `--prompt-cache-all`. The `./examples/chat-persistent.sh` script demonstrates this with support for long-running, resumable chat sessions. To use this example, you must provide a file to cache the initial chat prompt and a directory to save the chat session, and may optionally provide the same variables as `chat-13B.sh`. The same prompt cache can be reused for new chat sessions. Note that both prompt cache and chat directory are tied to the initial prompt (`PROMPT_TEMPLATE`) and the model file.
397+
398+
```bash
399+
# Start a new chat
400+
PROMPT_CACHE_FILE=chat.prompt.bin CHAT_SAVE_DIR=./chat/default ./examples/chat-persistent.sh
401+
402+
# Resume that chat
403+
PROMPT_CACHE_FILE=chat.prompt.bin CHAT_SAVE_DIR=./chat/default ./examples/chat-persistent.sh
404+
405+
# Start a different chat with the same prompt/model
406+
PROMPT_CACHE_FILE=chat.prompt.bin CHAT_SAVE_DIR=./chat/another ./examples/chat-persistent.sh
407+
408+
# Different prompt cache for different prompt/model
409+
PROMPT_TEMPLATE=./prompts/chat-with-bob.txt PROMPT_CACHE_FILE=bob.prompt.bin \
410+
CHAT_SAVE_DIR=./chat/bob ./examples/chat-persistent.sh
411+
```
412+
394413
### Instruction mode with Alpaca
395414
396415
1. First, download the `ggml` Alpaca model into the `./models` folder

0 commit comments

Comments
 (0)