Make TLS & QUIC Pay-for-Play #47209

amcasey · 2023-03-15T00:37:02Z

This draft PR is intended as context for a discussion and is not ready to commit.

As discussed offline, the way to think about disabling TLS is that Kestrel won't do things automatically for you, but there are things you can do yourself even without opting in. I think of it as disabling TLS configuration - Kestrel won't call UseHttps on your behalf (e.g. because you have specified an https address) or load certificates from IConfiguration or CertificateManager. However, you remain free to specify a certificate in code and call UseHttps explicitly. This is enough to allow the cryptography types to be trimmed and seems more defensible than having Kestrel throw whenever you get to close to TLS.

In contrast, QUIC support is either on or off.
Opting out effectively means that Kestrel won't call UseQuic on your behalf.

Big changes:

Introduce UseKestrelSlim, which doesn't use QUIC or TLS configuration.
Introduce UseHttpsConfiguration, which opts in to TLS configuration (QUIC uses the existing UseQuic extension method).
Split out the TLS portion of KestrelConfigurationLoader into a DI component, ITlsConfigurationLoader.
Break AddressBinder's direct dependency on UseHttps (called for addresses that parse as https).
Split TransportManager into multiplexed and non-multiplexed DI components.
Make KestrelServerImpl a DI component.
Make ServiceContext a DI component.
Throw when a failure is caused by a missing opt-in. (In particular, distinguish between lack of QUIC support and lack of QUIC opt-in.)

Open Questions:

Is there a better way to have UseQuic and UseHttpsConfiguration coordinate across assembly boundaries than introducing types like MultiplexedConnectionMarkerService?
Is this a sensible place to cut off TLS support?

Originally based on @eerhardt's bffa15c

@eerhardt

This draft PR is intended as context for a discussion and is not ready to commit. As discussed offline, the way to think about disabling TLS is that Kestrel won't do things automatically for you, but there are things you can do yourself _even without opting in_. I think of it as disabling TLS _configuration_ - Kestrel won't call `UseHttps` on your behalf (e.g. because you have specified an https address) or load certificates from `IConfiguration` or `CertificateManager`. However, you remain free to specify a certificate in code and call `UseHttps` explicitly. This is enough to allow the cryptography types to be trimmed and seems more defensible than having Kestrel throw whenever you get to close to TLS. In contrast, QUIC support is either on or off. Opting out effectively means that Kestrel won't call `UseQuic` on your behalf. Big changes: 1. Introduce `UseKestrelSlim`, which doesn't use QUIC or TLS configuration. 1. Introduce `UseHttpsConfiguration`, which opts in to TLS configuration (QUIC uses the existing `UseQuic` extension method). 1. Split out the TLS portion of `KestrelConfigurationLoader` into a DI component, `ITlsConfigurationLoader`. 1. Break `AddressBinder`'s direct dependency on `UseHttps` (called for addresses that parse as https). 1. Split `TransportManager` into multiplexed and non-multiplexed DI components. 1. Make `KestrelServerImpl` a DI component. 1. Make `ServiceContext` a DI component. 1. Throw when a failure is caused by a missing opt-in. (In particular, distinguish between lack of QUIC support and lack of QUIC opt-in.) Open Questions: 1. Is there a better way to have `UseQuic` and `UseHttpsConfiguration` coordinate across assembly boundaries than introducing types like `MultiplexedConnectionMarkerService`? 1. Is this a sensible place to cut off TLS support? Originally based on @eerhardt's dotnet@bffa15c

davidfowl · 2023-03-15T06:04:39Z

The slim name really bugs me. I think our current naming scheme of Add/Use{Thing}Core already exists and we should continue to use it. I’ll take a deeper look at the refactoring soon 👍🏾

amcasey · 2023-03-15T16:00:11Z

The slim name really bugs me. I think our current naming scheme of Add/Use{Thing}Core already exists and we should continue to use it. I’ll take a deeper look at the refactoring soon 👍🏾

I don't feel strongly about "slim", but I did consciously choose it over "core" because leaving out exactly these two things didn't feel "core" to me. I expect vigorous discussion. 😉

halter73 · 2023-03-16T00:16:20Z

src/Servers/Kestrel/Core/src/Internal/Infrastructure/IMultiplexedTransportManager.cs

+
+    Task StopAsync(CancellationToken cancellationToken);
+    Task StopEndpointsAsync(List<EndpointConfig> endpointsToStop, CancellationToken cancellationToken);
+}


Instead of adding IMultiplexedTransportManager and ITransportManager, could we register IMultiplexedHttpsFeatureFactory? I know it's internal, but it seems like the smaller change unless there's some other benefit to splitting things up.

internal interface IMultiplexedHttpsFeatureFactory { void PopulateHttpsFeatures(IFeatureCollection features, ListenOptions listenOptions); }

We could have the default implementation throw an InvalidOperationException similar to InvalidUseHttpsHelper.

Edit: It might make sense just to combine these:

internal interface IUseHttpsHelper { ListenOptions UseHttps(ListenOptions listenOptions); void PopulateHttpsFeatures(ListenOptions listenOptions, IFeatureCollection features); }

Or we could add an ListenOptions.Features and have UseHttps(ListenOptions listenOptions) populate it and keep the one method.

I'm a big fan of limiting the number of services even if they're internal.

Sorry, I'm not sure I understand this suggestion. (I had assumed I could go study IMultiplexedHttpsFeatureFactory, but I think you just invented it?) I think you might be suggesting that we add the pay-for-play functionality to a (the?) feature collection, rather than the DI container?

Implemented in #47454.

amcasey · 2023-03-21T21:41:04Z

@davidfowl I'm getting ready to polish and add tests. Any major notes before I do?

amcasey · 2023-03-22T19:53:23Z

@halter73 How's 36e2106 as a way to enable explicit UseHttps calls without UseHttpsconfiguration?

halter73 · 2023-03-22T20:21:59Z

src/Servers/Kestrel/Core/src/ListenOptionsHttpsExtensions.cs

@@ -163,33 +163,28 @@ public static ListenOptions UseHttps(this ListenOptions listenOptions, Action<Ht
    {
        ArgumentNullException.ThrowIfNull(configureOptions);

+        // TODO (acasey): apply to other overloads?
+        listenOptions.KestrelServerOptions.EnableTlsConfigurationLoading();


// TODO (acasey): apply to other overloads?

Would we do this to enable configuring things like default SslProtocols from config? I think that makes sense. It seems like the least-breaking thing to do.

On all the other overloads, the actual cert will be unchanged by config, right?

Agreed that doing it in all overloads makes sense.

This covers exactly the overloads that don't say This does not use default certificates or other defaults specified via config or KestrelServerOptions.ConfigureHttpsDefaults(Action{HttpsConnectionAdapterOptions. I can't decide whether it feels more intuitive to make UseHttps imply UseHttpsConfiguration in all cases or to respect the fact that some overloads are independent of IConfiguration. I'm leaning towards the latter.

src/Servers/Kestrel/Core/src/KestrelServerOptions.cs

halter73 · 2023-03-22T21:08:06Z

src/Servers/Kestrel/Core/src/KestrelConfigurationLoader.cs

+            if (_loaded)
+            {
+                Reload();
+            }


Given that if this is called, we know that UseHttps() is going to read the default HTTPS config immediately via ApplyHttpsDefaults and ApplyDefaultCertificate, wouldn't it be better to always reload?

That might be enough to close #45801 with this PR. It's not as good as delaying the configuration reading all the way until binding, but it fixes the primary issue without breaking dotnet-monitor detecting the lack of default cert by catching InvalidOperationExceptions.

Suggested change

if (_loaded)

{

Reload();

}

Reload();

I'm not sure I follow - what would be the benefit of reloading here, rather than where we were about to do it anyway? When we were discussing #45801, I thought it was important to defer the code-based configuration, rather than to move up the IConfiguration-based configuration.

When we were discussing #45801, I thought it was important to defer the code-based configuration, rather than to move up the IConfiguration-based configuration.

I still think it's better in principle to delay the code-based configuration to avoid extra config loading. But we tried that, and it broke dotnet-monitor.

I'm just trying to be realistic, and in the absence of an easy path forward to delay reading the configuration for UseHttps(), recognize that moving up the configuration loading in this case where we know it's going to be eagerly read is still better than not despite knowing it will get reloaded again later.

In a perfect world, we wouldn't be eagerly reading config, but we are. If and when we do delay reading config, I fully support removing the extraneous load.

In that case, I think I'd rather leave it the way it is and then make a second change after this is merged removing the _loaded check and explaining how that mitigates #45801. I think that will be much clearer if/when someone tries to clean it up.

Are we going to test this by manually calling kso.ConfigurationLoader!.Load() before calling kso.Listen(..., lo => UseHttps())? Because that's the only way this code ever does anything at all, right?

It seems weird to me to touch this code but not try to make the common case work when it's less code just because it happens to fix another existing bug. That said, if you think this is going to cause controversy or unnecessary friction with this PR, fine. I just don't want to leave this completely unfixed in .NET 8, and putting it off always increases that risk.

I wasn't aware that _loaded would always be false here. Can I assert that instead of checking it?

I thought the case being fixed in #45801 was explicitly calling UseHttps when running in a docker container. I assumed that was less common that configuring it by setting the address because I didn't think it had ever worked and it would have been weird to ship with the common case broken. But maybe you meant something else by "common case"?

Besides the logical tidiness of having separate commits for separate intended outcomes, the reason I'm inclined to separate them is that both the pay-for-play change and the configuration-loading-sequence change seem likely to have unintended consequences, especially in codebases with very specific expectations, and I think investigating and fixing those problems will be easier if the commits are separate.

src/Servers/Kestrel/Core/src/ListenOptionsHttpsExtensions.cs

…ITlsConfigurationLoader

amcasey · 2023-03-23T16:29:25Z

I have two concerns with this approach to making UseHttps just work.

It only lights up part of UseHttpsConfiguration. In particular, we still won't automatically call UseHttps from AddressBinder and we'll be missing the cert functionality for HTTP/3 (which could matter if the user has already called UseQuic).
There's no corresponding way to light up UseQuic, e.g. because we've detected use of Protocol.Http3.

…SvcInResponseHeaders

amcasey · 2023-03-24T23:24:47Z

src/Servers/Kestrel/Transport.Quic/src/WebHostBuilderQuicExtensions.cs

@@ -20,15 +19,11 @@ public static class WebHostBuilderQuicExtensions
    /// <returns>The <see cref="IWebHostBuilder"/>.</returns>
    public static IWebHostBuilder UseQuic(this IWebHostBuilder hostBuilder)
    {
-        if (QuicListener.IsSupported)
+        return hostBuilder.ConfigureServices(services =>


As @JamesNK mentioned here, this breaks all the checks that assume the presence of a multiplexed factory indicates QUIC support and a transport that can potentially bind (e.g. here)

That's why we're going to use check IConnectionListenerFactorySelector.CanBind rather than the presence of the IMultiplexedConnectionListenerFactory to determine whether QUIC is supported where necessary, right?

How? Wouldn't you need to pass in an endpoint?

The check is here, right? With a similar check just below it in the same method.

aspnetcore/src/Servers/Kestrel/Core/src/Internal/KestrelServerImpl.cs

Lines 184 to 187 in 7aca303

if (hasHttp3 && _multiplexedTransportFactories.Count == 0 && !(hasHttp1 || hasHttp2))

{

throw new InvalidOperationException("This platform doesn't support QUIC or HTTP/3.");

}

So options.EndPoint, I'd assume. If none of the _multiplexedTransportFactories implement IConnectionListenerFactorySelector and say they can bind to the endpoint, then HTTP/3 isn't going to work.

amcasey · 2023-04-04T17:38:48Z

Superseded by #47454, which attempts to incorporate the feedback on this PR.

ghost added the area-runtime label Mar 15, 2023

halter73 reviewed Mar 16, 2023

View reviewed changes

Drop MultiplexedConnectionMarkerService

8041dd1

Add support for UseHttps() without explicit opt-in

36e2106

halter73 reviewed Mar 22, 2023

View reviewed changes

Correct check in ApplyDefaultCertificate

3d6983b

halter73 reviewed Mar 22, 2023

View reviewed changes

src/Servers/Kestrel/Core/src/ListenOptionsHttpsExtensions.cs Outdated Show resolved Hide resolved

amcasey added 3 commits March 22, 2023 16:50

Update comments in ListenOptionsHttpsExtensions.cs

91e7e55

Retain TLS enablement when KestrelConfigurationLoader is replaced

c20c2e1

Eliminate Action<KestrelConfigurationLoader> in favor of storing the …

a6d600c

…ITlsConfigurationLoader

amcasey mentioned this pull request Mar 23, 2023

Remove premature QuicListener.IsSupported check #44755

Closed

Fix AltSvc_Http1And2And3EndpointConfigured_NoMultiplexedFactory_NoAlt…

327ebdb

…SvcInResponseHeaders

amcasey commented Mar 24, 2023

View reviewed changes

amcasey mentioned this pull request Mar 27, 2023

Make TLS & QUIC Pay-for-Play (Redux) #47454

Merged

amcasey closed this Apr 4, 2023

amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Jun 6, 2023

	if (hasHttp3 && _multiplexedTransportFactories.Count == 0 && !(hasHttp1 \|\| hasHttp2))
	{
	throw new InvalidOperationException("This platform doesn't support QUIC or HTTP/3.");
	}

Make TLS & QUIC Pay-for-Play #47209

Make TLS & QUIC Pay-for-Play #47209

Uh oh!

Conversation

amcasey commented Mar 15, 2023

Uh oh!

davidfowl commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amcasey commented Mar 15, 2023

Uh oh!

halter73 Mar 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amcasey commented Mar 21, 2023

Uh oh!

amcasey commented Mar 22, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

amcasey commented Mar 23, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

halter73 Mar 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amcasey commented Apr 4, 2023

Uh oh!

Uh oh!

davidfowl commented Mar 15, 2023 •

edited

Loading

halter73 Mar 16, 2023 •

edited

Loading

halter73 Mar 27, 2023 •

edited

Loading