field-value rule does not allow single field-vchar surrounded by whitespace [was: fix field-value ABNF]

[reschke]

see https://www.rfc-editor.org/errata/eid4189

[annevk]

In the range 0x00-0x7F Chrome, Edge, and Firefox all seem to only do something special with 0x00, 0x0A, and 0x0D. Everything else goes. Chrome and Firefox also apply this to the range 0x7F-0xFF. Edge starts transmitting UTF-8 bytes for those "code points" instead... Safari Technology Preview restricts 0x00-0x1F and 0x7F.

It seems easiest to simply not have constrains here.

[annevk]

Note that in whatwg/xhr#165 I've been trying to get browsers to align on rejecting responses where a header value contains a 0x00. Not much progress thus far, but I do still expect success eventually.

[reschke]

Clarification to @annevk - this issue is about an ABNF issue. If you want to discuss restricting the set of allowed code points, please do that in a seperate ticket.

[reschke]

Citing the erratum (https://www.rfc-editor.org/errata/eid4189):

the field-value rule given in Section 3.2 will not recognize several strings recognized by specific header rules.

Examples:
- ", , ," recognized by legacy list rule
- "abrowser/0.001 (C O M M E N T)" recognized by User-Agent rule
- "gzip , chunked" recognized by Transfer-Encoding rule
- etc.

General Problem:
the specified field-value rule does not allow single field-vchar surrounded by whitespace anywhere

Further Notes:
-what the authors propably wanted to say:
a string of octets is a field-value if, and only if:
-it is *( field-vchar / SP / HTAB / obs-fold )
-if it is not empty, it starts and ends with field-vchar

-the suggested correction was designed according to these criteria

[reschke]

Proposed change from https://lists.w3.org/Archives/Public/ietf-http-wg/2015JanMar/0403.html:

Section: 3.2

Original Text
-------------
     field-name     = token
     field-value    = *( field-content / obs-fold )
     field-content  = field-vchar [ 1*( SP / HTAB ) field-vchar ]
     field-vchar    = VCHAR / obs-text

     obs-fold       = CRLF 1*( SP / HTAB )
                    ; obsolete line folding
                    ; see Section 3.2.4

Corrected Text
--------------
     field-name     = token
     field-value    = *( field-content / obs-fold )
     field-content  = field-vchar [ 1*( SP / HTAB / field-vchar ) field-vchar ]
     field-vchar    = VCHAR / obs-text

     obs-fold       = OWS CRLF RWS
                    ; obsolete line folding
                    ; see Section 3.2.4

...where RWS == 1*( SP / HTAB )

[annevk]

@reschke what makes you say that? Is 0x00 allowed?