Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GEP-3155: Complete Backend mTLS Configuration #3180

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
9f99d93
Initial commit
mkosieradzki Jul 2, 2024
c19be24
Pending changes exported from your codespace
mkosieradzki Jul 2, 2024
cd2fe15
Update geps/gep-3155/index.md
mkosieradzki Jul 11, 2024
b5f5112
Merge branch 'kubernetes-sigs:main' into codespace-obscure-space-zebr…
mkosieradzki Jul 22, 2024
cf4906d
Merge branch 'kubernetes-sigs:main' into codespace-obscure-space-zebr…
mkosieradzki Jul 22, 2024
edbf961
Initial commit
mkosieradzki Jul 2, 2024
9c29265
Initial commit
mkosieradzki Jul 2, 2024
42498db
Initial commit
mkosieradzki Jul 2, 2024
f25dbbc
Initial commit
mkosieradzki Jul 2, 2024
37590b9
Fix enum values
mkosieradzki Jul 22, 2024
3408b5a
Initial commit
mkosieradzki Jul 2, 2024
15fdfd9
Pending changes exported from your codespace
mkosieradzki Jul 2, 2024
bc3183e
Fix indentation
mkosieradzki Jul 22, 2024
ba3084b
Fix indentation
mkosieradzki Jul 22, 2024
6d1b13c
Update geps/gep-3155/index.md
mkosieradzki Jul 23, 2024
a2d458b
Update geps/gep-3155/index.md
mkosieradzki Jul 23, 2024
912f965
Merge branch 'kubernetes-sigs:main' into codespace-obscure-space-zebr…
mkosieradzki Jul 30, 2024
b3dae17
Removed per-service override from the proposal
mkosieradzki Jul 30, 2024
0d84adf
Merge branch 'kubernetes-sigs:main' into codespace-obscure-space-zebr…
mkosieradzki Jul 31, 2024
ef5b22b
- Reverted back changes necessary for the GatewaySpec
mkosieradzki Jul 31, 2024
bb48234
Update geps/gep-3155/index.md
mkosieradzki Aug 1, 2024
c46bb1e
Update geps/gep-3155/index.md
mkosieradzki Aug 1, 2024
308c7b0
Update geps/gep-3155/index.md
mkosieradzki Aug 1, 2024
7426dd8
Update geps/gep-3155/index.md
mkosieradzki Aug 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 198 additions & 0 deletions geps/gep-3155/index.md
mkosieradzki marked this conversation as resolved.
Show resolved Hide resolved
Copy link

@jaishals jaishals Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @mkosieradzki , if we specify client cert on Gateway then can you please share some insights as to how would the scenarios where there are services which do not need MTLS but need only End2End TLS coexist in the same Gateway -> Route configurations.

There could be a route targeting service (music) which needs MTLS and another route targeting service (shopping) which only needs End to End TLS.

These scenarios would not work on same Gateway if we use MTLS to all the services

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the distinction here is that we're providing a client cert at the Gateway level that should be presented if it's requested by the backend. @mkosieradzki had previously included a per-Service override for this config in BackendTLSPolicy. We ran into issues there where the personas attached to BackendTLSPolicy weren't particularly clear (see #3226 to chime in on that discussion), so this iteration of the GEP will leave per-Service overrides out, but I think that's still very much a longer term goal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There could be a route targeting service (music) which needs MTLS and another route targeting service (shopping) which only needs End to End TLS.

To my best understanding whether the Gateway will send the certificate to the Backend or not depends on the backend server configuration, i.e. whether it will send CertificateRequest message during the handshake as per https://datatracker.ietf.org/doc/html/rfc5246#section-7.3 or not.

Therefore if backend is not configured to request a client certificate, the Gateway will not send it, even it is configured. My original rationale for adding per-service overrides was about corner cases:

  • where server application expects a certificate issued by a different CA
  • where server application can work in two modes: if the cert is available or not, and lack of certificate is perfectly fine for the server (and causes fallback to a different authentication method).

I think we should revisit this scenario as soon as we figure out the #3226.

Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
# GEP-3155: Complete Backend mutual TLS Configuration

* Issue: [#3155](https://github.com/kubernetes-sigs/gateway-api/issues/3155)
* Status: Implementable

## TLDR

This GEP aims to complete the configuration required for Backend mutual TLS in Gateway
API. This includes the following new capabilities:

1. Configuration for the client certificate Gateways should use when connecting
to Backends
1. Ability to specify SANs on BackendTLSPolicy
1. Add TLS options to BackendTLSPolicy to mirror TLS config on Gateways

## Goals

* Add sufficient configuration that basic mutual TLS is possible between Gateways and
Backends
* Enable the optional use of SPIFFE for Backend mutual TLS

## Non-Goals

* Define how automatic mTLS should be implemented with Gateway API

## Introduction

This is a wide ranging GEP intending to cover three additions to the API that all
have a shared goal - enabling backend mutual TLS with Gateway API. Although this
specific GEP focuses on manual configuration across the board, the hope is that
it will also enable higher level automation to simplify this process for users.

## API

### Client Certs on Gateways

A key requirement of mutual TLS is that the Gateway can provide a client cert to the
backend. This adds that configuration to both Gateway and Service (via
BackendTLSPolicy).

#### Gateway-level (Core support)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is missing an API example - seems to be suggesting the addition of a new field under the Gateway spec stanza https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.Gateway?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this should have a quick sketch of what we need to add to Gateway.

Specifying credentials at the gateway level is the default operation mode, where all
backends will be presented with a single gateway certificate. Per-service overrides are
subject for consideration as the future work.

**1. Add a new `BackendTLS` field at the top level of Gateways**

```go
type GatewaySpec struct {
// BackendTLS configures TLS settings for when this Gateway is connecting to
// backends with TLS.
BackendTLS GatewayBackendTLS `json:"backendTLS,omitempty"'
}
type GatewayBackendTLS struct {
// ClientCertificateRef is a reference to an object that contains a Client
// Certificate and the associated private key.
//
// References to a resource in different namespace are invalid UNLESS there
// is a ReferenceGrant in the target namespace that allows the certificate
// to be attached. If a ReferenceGrant does not allow this reference, the
// "ResolvedRefs" condition MUST be set to False for this listener with the
// "RefNotPermitted" reason.
//
// ClientCertificateRef can reference to standard Kubernetes resources, i.e.
// Secret, or implementation-specific custom resources.
//
// This setting can be overriden on the service level by use of BackendTLSPolicy.
ClientCertificateRef SecretObjectReference `json:"clientCertificateRef,omitempty"`
}
```

### SANs on BackendTLSPolicy

This change enables the backend certificate to have a different identity than the SNI
(both are currently tied to the hostname field). This is particularly useful
when using SPIFFE, which relies on URI Subject Names which are not valid SNIs
as per https://www.rfc-editor.org/rfc/rfc6066.html#section-3.

In such case either connection properties or an arbitrary SNI, like cluster-local
service name could be used for certificate selection, while the identity validation
robscott marked this conversation as resolved.
Show resolved Hide resolved
will be done based on SubjectAltNames field.
robscott marked this conversation as resolved.
Show resolved Hide resolved

When specified, the certificate served from the backend MUST have at least one Subject
Alternate Name matching one of the specified SubjectAltNames.



**1. Add a new `SubjectAltNames` field to `BackendTLSPolicyValidation`**

```go
type BackendTLSPolicyValidation struct {
// SubjectAltNames contains one or more Subject Alternative Names.
// When specified, the certificate served from the backend MUST have at least one
// Subject Alternate Name matching one of the specified SubjectAltNames.
// +kubebuilder:validation:MaxItems=5
SubjectAltNames []SubjectAltName `json:"subjectAltNames,omitempty"`
}

// +kubebuilder:validation:Enum=Cookie;Header
type SubjectAltNameType string

const (
// HostnameSubjectAltNameType specifies hostname-based SAN.
//
// Support: Core
HostnameSubjectAltNameType SubjectAltNameType = "Hostname"

// URISubjectAltNameType specifies URI-based SAN, e.g. SPIFFE id.
//
// Support: Core
URISubjectAltNameType SubjectAltNameType = "URI"
)


type SubjectAltName struct {
// Type determines the format of the Subject Alternative Name. Always required.
Type SubjectAltNameType `json:"type"`

// Hostname contains Subject Alternative Name specified in DNS name format. Required when Type is set to Hostname, ignored otherwise.
Hostname v1.PreciseHostname `json:"hostname,omitempty"`

// URI contains Subject Alternative Name specified in URI format. Required when Type is set to URI, ignored otherwise.
URI string `json:"uri,omitempty"`
}
```

**2. Modify Spec for `BackendTLSPolicyValidation` `Hostname`**

Before:
```go
// 2. Hostname MUST be used for authentication and MUST match the certificate
// served by the matching backend.
```

After:
```go
// 2. Only if SubjectAltNames is not specified, Hostname MUST be used for
// authentication and MUST match the certificate served by the matching
// backend.
mkosieradzki marked this conversation as resolved.
Show resolved Hide resolved
```

### Allow per-service TLS settings BackendTLSPolicy

Gateway level TLS configuration already includes an `options` field. This has
been helpful for implementation-specific TLS configurations, or simply features
that have not made it to the core API yet. It would be similarly useful to have
an identical field on BackendTLSPolicy.

Examples:
- configuration options for vendor-specific mTLS automation
- restrictions on the minimum supported TLS version or supported cipher suites
Comment on lines +144 to +151
Copy link
Contributor

@youngnick youngnick Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to be on record as saying that I didn't like this field when we added it to Gateway, I don't like this field here, but I don't have a good enough reason to block any changes. Let's see if it ends up misused in the wild.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely agree with the hesitation here, but ultimately believe this is more helpful than not. There's such a wide range of TLS config possible that it seems unlikely that we'll ever cover all of the potential concepts in the core API. I think it's likely preferable to mirror the options map we have on Gateway Listeners than to leave out a form of extensibility here, though I'll admit that both options have risks.


```go
type BackendTLSPolicySpec struct {
// Options are a list of key/value pairs to enable extended TLS
// configuration for each implementation. For example, configuring the
// minimum TLS version or supported cipher suites.
//
// A set of common keys MAY be defined by the API in the future. To avoid
// any ambiguity, implementation-specific definitions MUST use
// domain-prefixed names, such as `example.com/my-custom-option`.
// Un-prefixed names are reserved for key names defined by Gateway API.
//
// Support: Implementation-specific
//
// +optional
// +kubebuilder:validation:MaxProperties=16
Options map[AnnotationKey]AnnotationValue `json:"options,omitempty"`
}
```

## Conformance Details

Conformance tests will be written to ensure the following:

1. When SubjectAltNames are specified in BackendTLSPolicy:

- The hostname field is still used as SNI, if specified
- A certificate with at least one matching SubjectAltName is accepted
- A certificate without a matching SubjectAltName is rejected

2. When a Client Certificate is specified on a Gateway:

- It is applied to all services.
- The appropriate status condition is populated if the reference is invalid

## Future work
This GEP does not cover per-service overrides for client certificate. This is mostly for two reasons:

- it supports only the niche use cases - it should be reconsidered in future
- in current model, where BackendTLSPolicy shares namespace with the service instead of the Gateway, there are non-trivial security implications to adding client certificate configuration at this level - therefore ownership and colocation of BackendTLSPolicy (Service vs Gateway) needs to be figured out first

## References

This is a natural continuation of
[GEP-2907](https://gateway-api.sigs.k8s.io/geps/gep-2907/), the memorandum GEP
that provided the overall vision for where TLS configuration should fit
throughout Gateway API.
13 changes: 13 additions & 0 deletions geps/gep-3155/metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: internal.gateway.networking.k8s.io/v1alpha1
kind: GEPDetails
number: 3155
name: Complete Backend mutual TLS Configuration
status: Implementable
authors:
- mkosieradzki
- robscott
relationships:
seeAlso:
- number: 2907
name: TLS Configuration Placement and Terminology
description: Will use some of the terminology defined by this GEP.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ nav:
- geps/gep-2649/index.md
- Implementable:
- geps/gep-995/index.md
- geps/gep-3155/index.md
- Experimental:
- geps/gep-1619/index.md
- geps/gep-1742/index.md
Expand Down