Escaping documentation for line protocol fields is not correct

[danielnelson]

The line protocol documentation reads:

For string field values use a backslash character \ to escape:

• double quotes "

However, according to the current implementation in InfluxDB, backslashes also need to be escaped. This makes another line in the section incorrect:

Line Protocol does not require users to escape the backslash character \.

URL for relevant page?

https://docs.influxdata.com/influxdb/v1.3/write_protocols/line_protocol_reference/#special-characters

What products and version are you using?

InfluxDB 1.3.4

Where did you look before opening the issue?

influxdata/influxdb#8728

[russorat]

I went through and tested all the inputs on that page and they all work in InfluxDB 1.3.6. I have added an additional example of how backslashes are interpreted by InfluxDB.

[danielnelson]

If backslash is the last character then you must escape it, to avoid escaping the quote:

temperature_str="too hot cold\"

[sqwishy]

The documentation states that you need to escape the space character on tag values.

For tag keys, tag values, and field keys always use a backslash character \ to escape:
commas ,
equal signs =
spaces

#1309 states that influx won't complain if you escape backslash characters, but this doesn't seem to be the case with tag values at all. If you try to escape them, it ignores that.

> insert foo,key=hello\ world v=0
> select * from foo;
name: foo
time                key         v
----                ---         -
1517956635336335149 hello world 0

> insert foo,key=\hello\ world v=0
> insert foo,key=\\hello\ world v=0
> select * from foo;
name: foo
time                key           v
----                ---           -
1517956635336335149 hello world   0
1517956648057229075 \hello world  0
1517956650713189940 \\hello world 0

This seems to throw things out of whack when it thinks your \ is escaping a space or something that isn't supposed to be escaped. Because you can't escape the escape character to get it to treat it like a regular character...

> insert foo,key=\\hello\ world\ v=0
ERR: {"error":"unable to parse 'foo,key=\\\\hello\\ world\\ v=0': invalid tag format"}
> insert foo,key=\\hello\ world\\ v=0
ERR: {"error":"unable to parse 'foo,key=\\\\hello\\ world\\\\ v=0': invalid tag format"}

Am I missing something or do backslash characters just not work at all for tag values.

Just escape the escape character. That solves this issues, doesn't it?

[danielnelson]

You can use escapes in tag values:

> insert foo,bar=b\ az value=42

One exception is that the measurement name, tag keys, tag values, and field keys cannot end with a \, it is impossible to encode this due to the optional escaping.

[sqwishy]

Yeah, that's what I show in the example above. Except the escaping isn't even really optional in tag values. If I have \\ it sees it as \\ rather than a single backslash. insert foo,key=\\hello\ world v=0 inserts 1517956650713189940 \\hello world 0. If it were optional it might work since we can just escape all the backslashes going in as I believe we can with field values.

Having it optional at all is confusing. I don't see the reason for it.

[russorat]

Chatted with @danielnelson . to simplify everyone's lives, let's just state in the docs that you should always escape backslashes. sorry for the confusion @sqwishy

[danielnelson]

To further clarify, in string fields you should always escape " and \. For other sections of line protocol you MUST only escape the values listed in the current docs. @sqwishy There is no good reason, we would fix this but it would break backwards compatibility.

[sqwishy]

As far as I can tell, you can't escape backslash characters for tag values. So it's impossible to insert a tag value of foo\. I think I'm testing on 1.2 so maybe this is different or I'm doing something wrong. Otherwise, if this is a feature, that should probably be stated in the docs or something.

[sqwishy]

it would break backwards compatibility

That's what major releases are for. Can you fix it in 1.5?

This still seems broken in 1.4.3.

> insert foo,bar=hello\ world v=0
> select * from foo
name: foo                                                                                                                                                                                      
time                bar         v                                                                                                                                                              
----                ---         -                                                                                                                                                              
1517959549318344814 hello world 0                                                                                                                                                              

> insert foo,bar=hello\ world\ v=0
ERR: {"error":"unable to parse 'foo,bar=hello\\ world\\ v=0': invalid tag format"}
> insert foo,bar=hello\ world\\ v=0
ERR: {"error":"unable to parse 'foo,bar=hello\\ world\\\\ v=0': invalid tag format"}
> insert foo,bar=\\hello\ world v=0
> select * from foo
name: foo                                                                                                                                                                                      
time                bar           v                                                                                                                                                            
----                ---           -                                                                                                                                                            
1517959549318344814 hello world   0                                                                                                                                                            
1517959573207691825 \\hello world 0                                                                                                                                                            

Is it possible to insert hello world\ as the tag value?

[danielnelson]

You can only escape ,, , and = in a tag value.

Is it possible to insert hello world\ as the tag value?

No, it is impossible. I wouldn't call it a feature, but we should still probably mention this in the docs.

That's what major releases are for. Can you fix it in 1.5?

Minor releases (1.4, 1.5, 1.6) add new functionality but do not break backwards compatibility. You would need to do a major release (2.0). BTW, there is an issue open about this: influxdata/influxdb#6037