Path does not match the request if contains encoded url

[UnoSD]

I opened this issue on the aspnet/HttpAbstractions repository as the HttpRequestFeature.Path property was returning an unescaped : which was originally a %3A in my request's URL:

http://localhost:5000/secrets/http%3A%2F%2Fwww.g -> /secrets/http:%252F%252Fwww.g

The double escaping is generated in the Path property, but I have been advised the un-escaping of the %3A is happening in Kestrel/WebListener and I believe it is a bug. I am happy to help fixing it if you could point me in the right direction of the code.

[cesarblum]

Unescaping %3A is by design. The bug is in PathString, where the % sign is being escaped as %25. PathString needs to be fixed to look ahead and leave %2F.

[UnoSD]

@CesarBS can you please explain why by design %3A is un-escaped and %2F is not? Or why they are not both left escaped?

[cesarblum]

The Path property in the feature should give you the unescaped path from the HTTP request. So any percent-encoded character will be unescaped there. %2F is an exception because it is the path separator and could make a request invalid if unescaped. Consider an app that serves requests like /resource/a%2Fb. That request is for resource a/b under /resource. If we unescaped %2F, that request would become /resource/a/b, which is requesting resource b under /resource/a.

Are you trying to access the raw request target exactly as the server saw it? There is a way to do that but we strongly recommend users don't do it.

[UnoSD]

Thanks for the reply; I'm using a middleware for Hmac authentication and I'm using the full URL as part of the signature hash. Being the original URL on the client: http://localhost:5000/secrets/http%3A%2F%2Fwww.g and obtaining instead http://localhost:5000/secrets/http:%2F%2Fwww.g on the server I have non-matching hashes.

So yes, I "need" the raw URL as the client and the server should be able to have access to the same string for hashing. I could just trim some data from the hash, but I did not appreciate it would have been difficult to have matching URL between client/server.

[cesarblum]

This is how you get the raw target:

httpContext.Features.Get<IHttpRequestFeature>().RawTarget

[Tratcher]

FYI @UnoSD, the client does not transmit the original url in the request. ASP.NET has to approximate it by re-assembling the fields that the client does send. E.g. the request line contains path and query, the Host header contains the host and port, and the scheme is implied by the connection protocol. Also, any intermediate (e.g. proxy) may alter these fields. You can't reliably reproduce the full raw url that the client requested, only something that should be semantically equivalent.

[UnoSD]

Thank you both; @Tratcher the requests will be always coming from the local machine so it should be fine, but since there is no guarantee the server can match the URL I might drop this from the signature.
EDIT: Changed my mind again as it seems quite common to use the absolute URI for the signature in Hmac implementations for web requests, not sure I want to break from the convention and make it more confusing for potential consumers.