Process escape sequences given in prompts

[DannyDaemonic]

In Linux we can use a bashism to inject newlines into the prompt:

./main -m models/7B/ggml-model.bin -n -1 --color -r "User:" --in-prefix " " --prompt $'User: Hi\nAI: Hello. I am an AI chatbot. Would you like to talk?\nUser: Sure!\nAI: What would you like to talk about?\nUser:'

But in Windows there's simply no way. This patch processes the escape sequence given in prompts.

In Linux and on Macs, it allows us a cleaner way of doing it:

./main -m models/7B/ggml-model.bin -n -1 --color -r "User:" --in-prefix " " --prompt 'User: Hi\nAI: Hello. I am an AI chatbot. Would you like to talk?\nUser: Sure!\nAI: What would you like to talk about?\nUser:'

And makes the same thing possible in Windows:

main.exe -m models\7B\ggml-model.bin -n -1 --color -r "User:" --in-prefix " " --prompt "User: Hi\nAI: Hello. I am an AI chatbot. Would you like to talk?\nUser: Sure!\nAI: What would you like to talk about?\nUser:"

[jxy]

I don't understand why we need this? You could have put the actual characters in the string. Now you have a list of back slash escaped sequence that users may or may not aware. Now what if you want to put a json string to the prompt?

[DannyDaemonic]

Sorry, I must be overlooking something. Could you give a more concrete example of what this broke?

You could have put the actual characters in the string. Now you have a list of back slash escaped sequence that users may or may not aware.

The example I gave before still works just the same. It injects the actual characters into the string and works just fine.

./main -m models/7B/ggml-model.bin -n -1 --color -r "User:" --in-prefix " " --prompt $'User: Hi\nAI: Hello. I am an AI chatbot. Would you like to talk?\nUser: Sure!\nAI: What would you like to talk about?\nUser:'

But now Windows users can do the same thing. I think this is much more likely than someone has a prompt they want to use that happens to have \n in it and they don't want a new line.

You mention json, but you don't have to escape your quotes. I just tried these and they worked:

./main -m models/7B/ggml-model.bin -n -1 --color -r  --prompt '{"name":"John", "age":30, "car":null}'

./main -m models/7B/ggml-model.bin -n -1 --color -r  --prompt '{"glossary":{"title":"example glossary","GlossDiv":{"title":"S","GlossList":{"GlossEntry":{"ID":"SGML","SortAs":"SGML","GlossTerm":"Standard Generalized Markup Language","Acronym":"SGML","Abbrev":"ISO 8879:1986","GlossDef":{"para":"A meta-markup language, used to create markup languages such as DocBook.","GlossSeeAlso":["GML","XML"]},"GlossSee":"markup"}}}}}'

It doesn't even silently eat the escape sequences when they don't mean anything. You can even still do this:

./main -m models/7B/ggml-model.bin -n -1 --color -r  --prompt "I feel :\ right now.`

and it will use exactly that string.

I guess maybe if your json text itself had an escape sequence in it then that could be interpreted as an actual newline, but surely that has to be a really uncommon use case for main?

Still, if this is a big issue, we could always make a --straight-prompt or --no-escape option, or even change this one back and use -e like echo does.

[ivanstepanovftw]

In powershell you can do `n

[ivanstepanovftw]

By merging this: do we want to support this feature in future if it won't work or will break someone's prompt?

[DannyDaemonic]

There seems to be ways to do it in most shells, except Windows Command Prompt or in bat files. But not everyone knows how and it's different everywhere.

I guess the other answer to the question "why?" was just for convenience. I think some people prefer to just have escape sequences and I have yet to see a prompt this breaks. I checked our example prompts and our example scripts and they all work.

The tweet example provided by vanstepanovftw still works. I don't know why you think you have to escape your quotes, this doesn't change that.

Like I said, I don't have a problem making a -e switch that copies echo functionally but some utils that are designed to work with text like sed just have this behavior as default. I just want to make sure people understand how this works before we add yet another option.

The json example works and the tweet example works.

Edit: He deleted the tweet example but it just had single quotes in it.

I don't think this does what people think it does.

[DannyDaemonic]

I think there's confusion over what this patch does. It just lets you use escape sequences like \n on the command line. It doesn't mean you have to escape your quotes. The only reason you'd even have to change your prompt is if it you're passing it in with -p and it contains the sequence \n but you don't want that to mean new line (or \t don't mean it to be a tab, etc).

[jxy]

First, any Bourne compatible shell would allow you to do

./main -m models/7B/ggml-model.bin -n -1 --color -r "User:" --in-prefix " " --prompt \
'User: Hi
AI: Hello. I am an AI chatbot. Would you like to talk?
User: Sure!
AI: What would you like to talk about?
User:'

without the modification introduce here. It's much clearer and simpler.

Second, the chosen escape sequence introduced in this PR is so arbitrary,

                switch (input[++i]) {
                     case 'n':  output.push_back('\n'); break;
                     case 't':  output.push_back('\t'); break;
                     case '\'': output.push_back('\''); break;
                     case '\"': output.push_back('\"'); break;
                     case '\\': output.push_back('\\'); break;
                     default:   output.push_back('\\');
                                output.push_back(input[i]); break;

Do you expect user to specifically remember the prompt is parsed with these five and only five escape sequences? It's different from C. It's different from JSON. It's different from Bourne Shell's builtin echo. It's different from Bash's $'str' ANSI-C quoting.

Third, it breaks the input/output parity, and breaks the compatibility between --prompt and --file. Now you always have to look at it twice before saving your prompt to a file. And we can't do fancy things like

</dev/null ./main -r "User:" -p "$(cat my-previous-chat-with-bob.txt; echo " Think again?" ; echo "Bob:")"

without double check if we have one of the escape sequence in the file. Wait, what are the escape sequence again?

[jxy]

One more rant of the choice of the escape sequence (I promise this is the last one).

./main -p "Running under bash, can you tell how many backslashes we are giving to the model: \\\""

[DannyDaemonic]

Programs that primarily deal with text will often escape sequences automatically: awk, sed, perl, printf, whiptail, etc. I think it's generally useful. Personally I prefer using a single \n as opposed to breaking my command onto a second line. I tried to use the same escape sequences from sed but it's GPL so I couldn't just copy the code, and I think it gets you into a grey area if you go and read the code before writing your own. Either way, I agree it's not clear which things you need to escape.

I also agree that escape sequences in general can obfuscate things. In your example, even without main processing escape sequences, I feel most people would have to pause and think about what's going on with the backslashes in your example. I think people who use escape sequences often would just see \\ followed by \", and assume the user's intention was to escape a quote, which is what would happen here.

I'd also argue the input output parity is still there because you could use -f to read a file you wrote, but I see what you mean about wondering if there's anything that would need to be escaped when doing so with --prompt. I don't think it was intended to be used for feeding in an entire file but I don't see any harm changing things so it still can be used like that.

I'll make a pull request where escapes are only enabled with the use of -e as is done with echo .