CertificateRefs should NOT be required when TLS mode is "Terminate"

[robscott]

What happened:
CEL validation currently requires that CertificateRefs are specified when TLS mode is set to "Terminate":

gateway-api/apis/v1/gateway_types.go

Line 378 in 195d988

// +kubebuilder:validation:XValidation:message="certificateRefs must be specified when TLSModeType is Terminate",rule="self.mode == 'Terminate' ? size(self.certificateRefs) > 0 : true"

What you expected to happen:
This validation should be removed as implementations should be able to support other ways to configure TLS certificates. (One example would be GKE, but I don't think it is a secure practice to require all TLS certificates to be configured within a cluster).

Thanks to @gauravkghildiyal for catching this one!

[robscott]

We discussed this issue at today's community meeting and there were some suggestions that CertificateRefs should be used for every reference to a TLS Certificate. In my opinion, that would not be a good practice. Let's consider the options implementations might consider when supporting references to certificates outside of Kubernetes:

1. Virtual Resource

With this approach, we use certificate references but just use a "virtual" (nonexistent) resource as a way to reference something outside of Kubernetes.

  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
        group: certificates.example-vendor.com
        kind: TLSCertificate
        name: store-example-com

Pros:

• No additional resources needed
• Reuses certificateRefs field

Cons:

• Likely confusing that corresponding TLSCertificate resource does not exist
• Someone could install a TLSCertificate CRD to cause extra confusion
• Sometimes external resource names are not valid Kubernetes resource names so this would not work in all cases

2. Custom Resource

This is the same as above, but instead of a virtual resource, we're creating a new Custom Resource as well:

  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
        group: certificates.example-vendor.com
        kind: TLSCertificate
        name: store-example-com
---
kind: TLSCertificate
apiVersion: certificates.example-vendor.com/v1
metadata:
  name: store-example-com
spec:
  certName: store-example-com

Pros:

• Certificate name can have different validation than a Kubernetes resource name so this can support more use cases

Cons:

• Unnecessarily verbose configuration
• More things that can break - the reference to the TLSCertificate can be invalid or the reference from the TLSCertificate can be invalid
• If multiple Gateways reference the same TLSCertificate Kubernetes resource, the blast radius of changes is awfully large (imagine if someone deleted a TLSCertificate resource that was used by multiple Gateways)
• If Gateways in different namespaces refer to the same certificate, you either need to involve ReferenceGrant to allow cross-namespace references or have duplicate TLSCertificate resources per namespace (note that cross-namespace references are not meaningful in this case because we're doing this to refer to resources outside of Kubernetes)

3. Custom TLS Option

Instead of using CertificateRefs, we can use a TLS option. Note that per the spec, "Options are a list of key/value pairs to enable extended TLS configuration for each implementation", so this feels like a reasonably appropriate use of the spec - referring to an implementation-specific concept.

  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      options:
        example-vendor.com/certificate-name: store-example-com

Pros:

• Most concise configuration available
• Avoids unnecessary intermediate resources
• Supports more extensive set of characters

Cons:

• Does not populate CertificateRefs
• Loosened CertificateRef validation could make it slightly harder to find configuration issues

In my opinion, this is the least bad option of them all. I think we very much need to support TLS Certificates that are defined outside of Kubernetes, and on today's call, I think there was broad agreement on that goal, so the real question is just "how". Although implementations can use any of the above approaches, I feel strongly that we should allow the third one with our validation.

As one final note, it's worth considering that some Gateways may rely on certificate generation where the Certificate is populated after the Gateway is defined. Removing this validation would also support that use case.

[arkodg]

im a -1 on removing the certificateRefs requirement when the mode is set to Terminate until there is a way to express intent by the user that the system is taking care of cert management on their behalf. This affects all other implementations that would like to surface the error early to the user if the field is empty.

As youve mentioned here, there is def a use case for the user to configure TLS Terminate w/o having to generate certs like autocert/ACME, prefer if we find a way to define this use case, and then modify the validation to allow for empty certificateRefs if this intent is defined

[robscott]

-1 on removing the certificateRefs requirement when the mode is set to Terminate until there is a way to express intent by the user that the system is taking care of cert management on their behalf.

I think the problem is that I'm not sure that it's possible to make all possible alternative cert sources portable. I personally believe that a Custom TLS Option is the least bad option available today, but there are also compelling arguments to be made for Gateway-wide annotations. I don't think we should try to fit every possible extension into a single pattern here.

This affects all other implementations that would like to surface the error early to the user if the field is empty.

I'm hoping that many other implementations would already have a fairly standard way to communicate configurations that the API allows but that the implementation does not support. Unfortunately one of the limitations of a portable API like this is that it's unavoidable that the API will allow config that some implementations simply have no path to support.

[arkodg]

rethinking, I think its reasonable to make the above change, would prefer if this work also enhances to doc string to explain why (or when) its okay to not populate the certRefs field.
This would also unblock implementations from supporting TLS Termination for the user's hostname w/o having the user to define the certRefs here

[youngnick]

Okay, to summarize, the requested change is:

• Remove the CEL validation that enforces that certRefs must be specified when the TLS mode is set to Terminate.

Allowing this does allow for more "magic" implementations (for lack of a better word) - but that exact magic is very likely to be not portable.

I definitely prefer the clarity of requiring the reference, but not enough to be a hard "no" on this suggested change.

However, if we are going to do this, I think we also need to do two things:

• Record the expected behaviors and explicitly call this behavior out as ImplementationSpecific.
• Start a discussion around writing down the use cases we expect here, and see if we can start trying to find some common ground so that we can ensure we don't end up with wildly divergent implementation methods. I think my desired outcome here is, at the very least, some recommendations about ways to handle the various use cases with the tools we have today.