[EPIC]: HTTPS and Certificate Handling in Kestrel

[davidfowl]

This epic tracks all the work required in Kestrel to improve certificate management.

• - Support specifying a full certificate chain in HttpsConnectionAdapterOptions - Support specifying a full certificate chain in HttpsConnectionAdapterOptions #21513
  • - SSLStream should support taking a pre-validated immutable full certificate chain - SSLStream should support taking a pre-validated immutable full certificate chain runtime#35844 - SslStream API improvements for enhanced use cases  runtime#37933
• - Add support for PEM format SSL certificates - Add support for PEM format SSL certificates #4706
  • Add easy way to create a certificate from a multi-PEM or cert-PEM + key-PEM runtime#31944
• - Add support for loading entire key chains from PFX and PEM files. [Kestrel] Certificate chains support #23623
• - Consider support for SNI certificate selection from X509Store - Consider support for SNI certificate selection from X509Store #21300
• - ServerCertificateSelector callback needs to be async - ServerCertificateSelector callback needs to be async, otherwise any async work in callback may cause deadlock with threadpool (eg. logging failure to disk). #20981
• - Enable SslStream server options per-hostname - Enable SslStream server options per-hostname (in callback) runtime#31097
• - Support SNI via config - Support SNI via config #15144
• OCSP Stapling Support - OCSP Stapling support for Kestrel #4762
  • OCSP stapling support can be used to optimize TLS connections runtime#33377
• - Configure certificate by thumbprint - Configure certificate by thumbprint #4385

  • The problem with thumbprints is it'll allow you to choose an expired certificate. Using subject names is better because it will pull a valid one (assuming it exists), with the longest validity period. Thumbprint selection should die.

• - Cache certificate validation results in Certificate Auth Performance improvements for Client Certificate auth #12324
• - Enable setting the ClientCertificateMode from an appsettings.json file - Setting the ClientCertificateMode Kestrel server option from an appsettings.json file. #18660

[davidfowl]

cc @wfurt @karelz @bartonjs @vcsjones

[blowdart]

Do you want to add caching of certificate validation for certificate auth? #12324

[davidfowl]

Yea let me add that

[blowdart]

I added it because you didn't :p

[blowdart]

[webprofusion-chrisc]

I don't know if it's already tracked (or considered out of scope) but a common pattern for certificate use is to fetch them from a broker such as a central cert service or secrets store (using any method - http api, ACME etc - but this may be a very long async wait) then re-fetch and re-apply them as expiry looms. The local X509Store (if any) may just be considered a local cache.

It would be ideal if a standard interface for cert request & renewal could be provided (with custom providers for specific scenarios/implementations).

All domain/host certs have expiry (which can be hours away or weeks away, but will happen during app uptime), users may wish to start to request renewal after a fixed time period e.g. every 30 days or a set period before cert expiry, e.g. 48 hrs before expiry). Initial renewal failures may be transient but repeated failures require escalation.

Fetching may fail if the client no longer has permission to request the cert for that domain or other validation has failed, therefore the service would continue to use the expiring/expired certificate and retry etc.

As a service may have multiple domain/port bindings, the state of a certificate will vary per binding (some may be closer to expiry than others, may be refreshed using other means). Certs may be issued by public CAs or internal CAs (per binding). This is somewhat related to #21300 and #20981 but not completely and is really just a question of whether this scenario should be a concern for the core framework or left entirely to third-party middleware.

[davidfowl]

That’s pretty much why we leave policy out of the framework and push users to a callback model. That will be what we enable before doing anything more advanced. We may be able to integrate more as we get a feel for what customers do with it and if it’s common enough and difficult, we could bake something in