@@ -4,14 +4,25 @@ MODEL="./models/13B/ggml-model-q4_0.bin"
4
4
USER_NAME=" ${USER_NAME:- User} "
5
5
AI_NAME=" ${AI_NAME:- ChatLLaMa} "
6
6
7
- echo user_name: $USER_NAME
8
- echo ai_name: $AI_NAME
7
+ # Adjust to the number of CPU cores you want to use.
8
+ N_THREAD=" ${N_THREAD:- 8} "
9
+ # Number of tokens to predict (made it larger than default because we want a long interaction)
10
+ N_PREDICTS=" ${N_PREDICTS:- 1024} "
9
11
10
- GEN_OPTIONS=" --ctx_size 4096 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --repeat_penalty 1.17647"
12
+ # Note: you can also override the generation options by specifying them on the command line:
13
+ # For example, override the context size by doing: ./chatLLaMa --ctx_size 2048
14
+ GEN_OPTIONS=" ${GEN_OPTIONS:- --ctx_size 4096 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --repeat_penalty 1.17647} "
11
15
12
- ./main -m " $MODEL " -t 8 -n 1024 $GEN_OPTIONS --color -i -r " ${USER_NAME} :" -p \ "
16
+ # shellcheck disable=SC2086 # Intended splitting of GEN_OPTIONS
17
+ ./main $GEN_OPTIONS \
18
+ --model " $MODEL " \
19
+ --threads " $N_THREAD " \
20
+ --n_predict " $N_PREDICTS " \
21
+ --color --interactive \
22
+ --reverse-prompt " ${USER_NAME} :" \
23
+ --prompt "
13
24
Text transcript of a never ending dialog, where ${USER_NAME} interacts with an AI assistant named ${AI_NAME} .
14
- ${AI_NAME} is helpful, kind, honest, good at writing and never fails to answer ${USER_NAME} ’s requests immediately and with precision.
25
+ ${AI_NAME} is helpful, kind, honest, friendly, good at writing and never fails to answer ${USER_NAME} ’s requests immediately and with precision.
15
26
There are no annotations like (30 seconds passed...) or (to himself), just what ${USER_NAME} and ${AI_NAME} say alound to each other.
16
27
The dialog lasts for years, the entirety of it is shared below. It's 10000 pages long.
17
28
The transcript only includes text, it does not include markup like HTML and Markdown.
0 commit comments