At what level does it make sense to deal with URLs that contain unicode hostnames

[jhudsoncedaron]

If you tell me the browser I will be quite able to tell you you're wrong because I'm looking at a case where the browser is not involved.

I ended up with a configuration that looked like:

Application Server: https://🦍:444/
Auth Server: https://🦍:10443/

The call to the application server worked directly, but the redirect to the Auth server didn't. Nice big log stacktrace but it boils down to RedirectResult() accepted a URL with a unicode hostname but caused the request to error out later with System.InvalidOperationException: Invalid non-ASCII or control character in header: 0xD83E with the only piece of our code on the stack is the mitigation for compressed HTTPS leak.

[2021-10-06 13:12] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[2021-10-06 13:12]       Connection id "0HMC90M8308ND", Request id "0HMC90M8308ND:00000002": An unhandled exception was thrown by the application.
[2021-10-06 13:12]       System.InvalidOperationException: Invalid non-ASCII or control character in header: 0xD83E
[2021-10-06 13:12]          at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.ThrowInvalidHeaderCharacter(Char ch)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.ValidateHeaderValueCharacters(StringValues headerValues)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponseHeaders.SetValueFast(String key, StringValues value)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.Microsoft.AspNetCore.Http.IHeaderDictionary.set_Item(String key, StringValues value)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Http.DefaultHttpResponse.Redirect(String location, Boolean permanent)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.RedirectResultExecutor.ExecuteAsync(ActionContext context, RedirectResult result)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.RedirectResult.ExecuteResultAsync(ActionContext context)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultAsync(IActionResult result)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]()
[2021-10-06 13:12]       --- End of stack trace from previous location ---
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
[2021-10-06 13:12]       --- End of stack trace from previous location ---
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
[2021-10-06 13:12]       --- End of stack trace from previous location ---
[2021-10-06 13:12]          at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
[2021-10-06 13:12]          at Cedaron.FormBuilder.Presentation.Infrastructure.RandomHeaderMiddleware.InsertRandomHeader(HttpContext context, Func`1 next) in F:\src\Cedaron.FormBuilder.Presentation\Infrastructure\RandomHeaderMiddleware.cs:line 20
[2021-10-06 13:12]          at Cedaron.FormBuilder.Presentation.Infrastructure.StatusCodePageMiddleware.Invoke(HttpContext context) in F:\src\Cedaron.FormBuilder.Presentation\Infrastructure\StatusCodePageMiddleware.cs:line 20
[2021-10-06 13:12]          at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.Invoke(HttpContext context)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.HandleException(HttpContext context, ExceptionDispatchInfo edi)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
[2021-10-06 13:12]          at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

But now we get to this. Should RedirectResult() have handled it? Should HttpResponse.Redirect()? Somebody else? The string came in from a config file where the unicode hostname is perfectly appropriate.

It almost feels like it should be passed through System.Uri but that is clearly not cut out for the job. It really wants a Uri.ChangeHostname() function but there's not one.

[Tratcher]

Prior discussion for Redirect and location header encoding: #4919. Most of those deal with the path, but the host is equally relevant.

In sort, Redirect assumes you've already encoded the url because double encoding it would be just as bad. That means you need to deal with the encoding when creating the redirect value. UriHelper.BuildAbsolute would properly encode a unicode host to punycode for the Location header. If you already have a Uri then you can use UriHelper.Encode.

We should add this as a duplicate in #4919.

[jhudsoncedaron]

I have to agree with close duplicate.