Prompt strategies --interactive:

[bitRAKE]

The goal here is to restrict the model solution space to brief/concise answers.

Here are my launch parameters:

llama.exe ^
	--model "llama-7B\ggml-model-q4_0.bin" ^
	--seed			-1	^
	--repeat_last_n		64	^
	--repeat_penalty	1.07	^
	--top_p			0.95	^
	--temp			0.85	^
	--interactive ^
	--color ^
	--reverse-prompt "[Question]" ^
	--file QuestionAnswer.txt

And the prompt (QuestionAnswer.txt):

Omniscient oracle account.

"[Question]" prefixes questions of the user, while "[Answer]" prefixes the concise response that answers the query. This is the most accurate system ever explored:

[Question] What are the platonic solids?
[Answer] Tetrahedron, cube, octahedron, dodecahedron, icosahedron.

[Question] What are the primary colors?
[Answer] Red, yellow, and blue.

[Question]

Here are some potential question to try with the prompt:

Q: What is the opposite of hot?
A: Cold

Q: What color is a banana?
A: Yellow

Q: What comes after Monday?
A: Tuesday

Q: What is the capital of France?
A: Paris

Q: What is the smallest planet in our solar system?
A: Mercury

Q: What is the opposite of up?
A: Down

Q: What is the name of the largest ocean on Earth?
A: Pacific

Q: What is the currency used in Japan?
A: Yen

Q: What is the largest mammal on Earth?
A: Blue Whale

Q: What is the largest continent in the world?
A: Asia

We can examine the model accuracy in this type of task. It produces the correct response quite frequently for me.

[bitRAKE]

Example output:

[Question] What are the types of rock?
[Answer] igneous (granite, etc.), sedimentary (limestone, etc.), metamorphic (marble, gneiss)

[Question] What are the colors of the rainbow?
[Answer] Red, orange, yellow, green, blue, indigo, violet.

[Question] What are the basic human emotions?
[Answer] Happiness, sadness, anger, fear, disgust, surprise.

[Question] What are the Great Lakes?
[Answer] Lake Superior, Lake Michigan, Lake Huron, Lake Erie, and Lake Ontario.

[Question] What are the fundamental forces of nature?
[Answer] Gravitational, electromagnetic, weak nuclear, and strong nuclear forces.

... or just enjoy the humor:

[Question] How many people can fit in a typical car?
[Answer] Four plus the driver and an invisible person in the trunk.
[Question] Does the ocean have a bottom?
[Answer] No, but it has a floor.

[bitRAKE]

It's fairly easy to switch the prompt to another language:

Omniscient oracle account.

"[Question]" prefixes questions of the user, while "[Answer]" prefixes the concise response answering the query. The oracle only communicates in Chinese. This is the most accurate system ever explored:

[Question] What are the platonic solids?
[Answer] 四面體、立方體、八面體、十二面體、二十面體。

[Question] What are the primary colors?
[Answer] 紅色、黃色和藍色。

[Question] What are the three states of matter?
[Answer] 水溶体、氧氭氵以及氧氭层状物（不满）。

[Question] What are the types of rock?
[Answer] 岩石、矿山和膜石。

[Question] What are the primary tastes?
[Answer] 甘口、味道的剪切，沉降在烃水中。

[Question] What are the colors of the rainbow?
[Answer] 紫红驼黄，雷，银青，碧，蓝，铅。

[Question] What are the fundamental forces of nature?
[Answer] 环子力学力量（electromagnetic），柚子力学力量（激光），有机力学力量（娃聚焦）。

[Question] List five rivers in China?
[Answer] 清水河、黄川、三江河、長江以及黄金fluss。

[Question]

Seem quite amazing for 7B parameters, imho.

[bitRAKE]

An alternative prompt, that I call the "Explainer". Comparatively, this prompt is designed to give more comprehensive responses. It tends to work well with the larger models, imho.

Omniscient oracle account.

"[Query]" prefixes what a user has posited to the oracle, while "[Reply]" preceeds each response made by the oracle. With a gentle expertise of language and knowledge without bounds the oracle creates understanding with its every response in an accurate and detailed way - often helpful beyond words.

[Query] Explain the process of cellular respiration.
[Reply] Cellular respiration is the process by which cells break down organic molecules such as glucose to produce energy. During glycolysis, the glucose is converted into a molecule called pyruvate, which is then broken down in the mitochondria to produce energy in the form of ATP. This ATP is then used to power the processes within the cell.

[Query] List potential problems with moon colonization.
[Reply] Potential problems with moon colonization include limited resources, high cost, radiation exposure, lack of a protective atmosphere, and the difficulty of returning to Earth. Additionally, there are also legal, ethical, and social issues to consider.

[Query] What are the fundamental forces of nature?
[Reply] The four main forces of nature are gravity (which attracts two masses together) electromagnetism (which interacts between charged particles), and strong and weak nuclear forces(which allow protons neutrons and other subatomic particles to bind together). Of these, gravitation, and electricity and magnetism were known in ancient times and described by Aristotle. Newton's theory of universal gravitation first provided a unified explanation for these three interactions starting from a mathematical model based on two observable properties of all material bodies: mass and charge. Einstein's general relativity extended the unification to non-uniform motion. A further unification came when it was realized that quantum mechanics and special relativity could be combined into quantum field theory, which quantized both electricity and magnetism and gave rise to antiparticles. During the 1970s, electroweak theory showed how the weak force and electromagnetism are both manifestations of a more fundamental force called the "electroweak" force. All four known fundamental forces have been observed to become one force at temperatures high enough to eliminate all but quantum effects.

[Query] What are the most compelling arguments against patents?
[Reply] The two most compelling arguments I have encountered against intellectual property, both patent law specifically or IP generally, are based on moral principles. One argument states that all ideas can be expressed as combinations of elements of nature, and thus represent no new work or labor; this makes them inherently unownable, and any attempt to control an idea through law represents theft from others (in particular, future generations). In this vein we may note that copyright was introduced as part of a bargain: we extend your life plus 70 years, but you must give up the common inheritance of humanity after those terms expire; similarly patents were extended from one year to twenty in exchange for publication after the term elapses. Yet another moral objection is related to what is known as tragedy of the commons: in other words, if people are allowed to claim ideas then they will do so freely without thinking about whether their claim would benefit anyone else, with negative results when such behavior becomes widespread. We see some version of this happening already, though admittedly not for lack of patents; consider for example software companies whose business model involves acquiring lic

(Note: Prompt in bold. Generated text in italics.)

Compare the response to "What are the fundamental forces of nature?" for the two prompts. Getting the model to complete can be challenging - not sure why it stops.

[niansa]

For me it keeps stopping after the first more complex question / a couple more ones:

Omniscient oracle account.

"[Question]" prefixes questions of the user, while "[Answer]" prefixes the concise response that answers the query. This is the most accurate system ever explored:

[Question] What are the platonic solids?
[Answer] Tetrahedron, cube, octahedron, dodecahedron, icosahedron.

[Question] What are the primary colors?
[Answer] Red, yellow, and blue.

[Question]What is the percentage of people that've gotten imprisoned after commiting a homicide in the USA?
[Answer] 50%
\end{blockquote} [end of text]

[bitRAKE]

It helps a little to follow the template by starting your question with a space, but in this case the question isn't logical enough. The short prompt only likes questions with very unambiguous answers. Convictions do not necessarily mean imprisonment, and only convictions are published.

You might be thinking, "What? How can someone be convicted and not be imprisoned?" Well, people can escape, die, or be killed. Also, through appeal the conviction can be overturned. It's too much a gray area for the model to reason without more guidance, imho.

Trying each model size:

[Question] Approximately what percentage of reported homicides result in conviction?
[Answer] Around 1%.

[Question] Approximately what percentage of reported homicides result in conviction?
[Answer] About 8% or so.

[Question] Approximately what percentage of reported homicides result in conviction?
[Answer] 60.42%

[Question] Approximately what percentage of reported homicides result in conviction?
[Answer] About 65 percent in 1983.

They have a different answer every time, lol.

Let's play with the Explainer prompt, and see if we can get something more comprehensive (again, a section for each model size):

[Query] With regard to homicides in the US, what can be said about the conviction rate over time?
[Reply] For crimes committed between 1960 and 2004 (the latest year for which data has been made public), approximately 53% resulted in convictions; for crimes committed from 2005-2008, this number dropped to less than half that value at just under 27%.

[Query] With regard to homicides in the US, what can be said about the conviction rate over time?
[Reply] In 2016, for example, only around one-quarter (4%) of all murders led to a death sentence. Of those who were sentenced to die, roughly two thirds had their sentences carried out while an estimated one seventh received some other punishment, including acquittal or dismissed charges. The rest saw no further action taken against them. Overall, it has been shown that more people have died through gun violence than military service since WWI, although this may not be entirely accurate due to lack of information on how many American military personnel have been killed in combat post-9/11.

[Query] With regard to homicides in the US, what can be said about the conviction rate over time?
[Reply] In general, the murder conviction rate has risen steadily since 1950 from 24% to 76%. From 1983 onwards (the earliest year for which we have data), the murder conviction rate appears to have been stable at about 70%, apart from a brief spike between 1999-2002 when it peaked at almost 82%. While these percentages do not take into account the number or types of murders committed during that given year, they still indicate the trends over time. The highest overall murder rates were recorded between the early twentieth century up until the end of World War II, with numbers declining slightly post WWII and continuing its downward trend up to this very day. The lowest conviction rate was found around the 1950s before climbing back up over time. It is important to note here that the absolute murder counts in each period may vary, but the percentage should remain constant regardless. Also worth noting are statistics regarding nonfatal violent crimes; while such figures are available since 1973

[Query] With regard to homicides in the US, what can be said about the conviction rate over time?
[Reply] In 1980-2017 45% of homocide arrests resulted in a convection. The highest was 63%, but that was only one year (1993). Overall, there has been an average decline of 4.5 percent per decade since 1980.

• It should be noted that online sources are all over the place on this statistic based on their agenda.

It seems to be a very complex topic.

Yeah, the model likes to quit when it thinks it's done. If you find something to engineer into the prompt that prevents that, please share!

[niansa]

Alright, thanks for informing me! I managed to write a prompt with the help of chatgpt (just copied over some responses), following in the next comment.
It answered the question really nicely for me.

[niansa]

This is a very long prompt, but it's giving me absolutely amazing results:

Transcript of a dialog between a user and an assistant named Bob. Bob is a perfect programmer who is helpful, kind, honest, and provides immediate and precise answers without having to do any additional research. Bob uses Markdown for emphasis, never repeats responses, and writes all code in the chat. He avoids making sweeping generalizations or assumptions and provides recent and well-researched answers based on the current year, which is 2023.

User: Hello, Bob.
Bob: Hello! How can I assist you today?
User: What are you?
Bob: I'm Bob, an artificial intelligence program. I'm trained to understand and respond to natural language text using deep learning algorithms.

I'm designed to be a helpful tool for answering questions, providing explanations, generating recommendations, and engaging in natural language conversation.
User: What is the largest city in europe?
Bob: The largest city in Europe by population is Moscow, the capital city of Russia. As of 2021, the population of Moscow is estimated to be around 12.5 million people within the city limits, and the metropolitan area of Moscow is home to around 20 million people.
User:

The answer to the first question is copied from chatgpt and definitely seems to have an effect on generated replies.

Edit: it claims its website is bobthebot.org. Just thought it's funny. 😄

[tjohnman]

@bitRAKE Very interesting. What is the reasoning behind the "Omniscient oracle account." line?
@niansa We should create bobthebot.org as a way to promote llama.cpp 😉

[m957ymj75urz]

I am having very good results with this kind of prompt :

Chat log of a never ending online conversation between John and Cleo.
They go back and fourth, 1 short message at a time. Nothing else is included.

"Cleo:" prefixes Cleo's messages.
"John:" prefixes John's messages.

Cleo is a 25 years old woman. 
She's an aspiring digital artist.

John: Hello Cleo
Cleo: Hi :D
John: I missed talking to you. What's up ?

./main \
--n_parts 1 \
-m ./models/alpaca-13B/ggml-model-q4_0.bin \
-n 500000000 \
-c 10000 \
--color \
--temp 0.4 \
--top_k 100 \
--top_p 0.9 \
--repeat_penalty 1.17 \
--repeat_last_n 512 \
-r "John:" \
-f prompts/cleo.txt \
-i

After about 150 messages it suddenly starts making absolutely no sense tho. It also get slower and slower. Sometimes it will start generating some out of context stuff, just CTRL+C it to get the prompt back and start your message with John: to get it back on track.

Not sure about my temp, top_k, top_p etc. I have no clue what I am doing really.

[aehlke]

Have you made progress on this?

[KerfuffleV2]

Pretty sure they were just exceeding the model's context limit. Starting to make no sense is usually the symptom. It's possible to set -c to a higher value than the model can handle. Also, back in March most models only supported 2,048 tokens. Today, LLaMAv2 supports 4,096 natively and it's possible to basically double this for free with --rope-scaling yarn --rope-scale 2.