Have kestrel filter out pseudo headers

[Tratcher]

Fixes #30417
6.0 rc2 candidate: interoperability

HTTP/2 & 3 use pseudo headers for special values like method, path, authority, scheme, etc., but they have lots of constraints about where they can be placed, only using reserved ones, etc.. These fields are also available elsewhere in the request.

https://tools.ietf.org/html/rfc7540#section-8.1.2.1

Pseudo-header fields are not HTTP header fields.

As a result, other http header APIs won't accept them as input. E.g. HttpRequestMessage, WebHeaderCollection, etc.. This means components that copy request headers from Kestrel have to filter these out. E.g. YARP, WCF, gateways, etc..

https://github.com/microsoft/reverse-proxy/blob/a3247ba1168325f46ab0ee4a24d3de1d9acdd853/src/ReverseProxy/Service/Proxy/HttpTransformer.cs#L49-L53

// Filter out HTTP/2 pseudo headers like ":method" and ":path", those go into other fields.
if (headerName.Length > 0 && headerName[0] == ':')
{
    continue;
}

Http.Sys and IIS also filter out pseudo headers.

Side note: I noticed that the new header properties don't support removal. I've updated them.

[davidfowl]

This is an existing bug in .NET 6 being fixed right?

[JamesNK]

Should this particular change go into 6.0 RC2?

[Tratcher]

Yes, and yes I plan to backport this entire PR to rc2.

[wtgodbe]

Seems reasonable, are there any interop tests we need to perform/add to make sure this plays nice with YARP or any internal components?

[Tratcher]

YARP has 6.0 integration tests that will cover this.

[Tratcher]

/backport to release/6.0

[github-actions]

Started backporting to release/6.0: https://github.com/dotnet/aspnetcore/actions/runs/1214277678

[Tratcher]

We identified a bug with this approach in 6.0. On line 221 above, TryValidateMethod is skipped if the method was sent via static hpack index. That means we fail to remove it from the collection.

#38581 fixed this in inadvertently in 7.0 by bulk clearing the set bits in the collection itself rather than clearing the individual properties.