Merge branch 'main' into docs/add-1.16-release-notes

hashicorp · Jun 13, 2023 · cf111ab · cf111ab
2 parents bb5bfba + 0ddafcf
commit cf111ab
Show file tree

Hide file tree

Showing 30 changed files with 2,728 additions and 161 deletions.
diff --git a/.changelog/17593.txt b/.changelog/17593.txt
@@ -0,0 +1,3 @@
+```release-note:bug
+docs: fix list of telemetry metrics
+```
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,70 @@
+## 1.16.0-rc1 (June 12, 2023)
+
+BREAKING CHANGES:
+
+* api: The `/v1/health/connect/` and `/v1/health/ingress/` endpoints now immediately return 403 "Permission Denied" errors whenever a token with insufficient `service:read` permissions is provided. Prior to this change, the endpoints returned a success code with an empty result list when a token with insufficient permissions was provided. [[GH-17424](https://github.com/hashicorp/consul/issues/17424)]
+* peering: Removed deprecated backward-compatibility behavior.
+Upstream overrides in service-defaults will now only apply to peer upstreams when the `peer` field is provided.
+Visit the 1.16.x [upgrade instructions](https://developer.hashicorp.com/consul/docs/upgrading/upgrade-specific) for more information. [[GH-16957](https://github.com/hashicorp/consul/issues/16957)]
+
+SECURITY:
+
+* audit-logging: **(Enterprise only)** limit `v1/operator/audit-hash` endpoint to ACL token with `operator:read` privileges.
+
+FEATURES:
+
+* api: (Enterprise only) Add `POST /v1/operator/audit-hash` endpoint to calculate the hash of the data used by the audit log hash function and salt.
+* cli: (Enterprise only) Add a new `consul operator audit hash` command to retrieve and compare the hash of the data used by the audit log hash function and salt.
+* cli: Adds new command - `consul services export` - for exporting a service to a peer or partition [[GH-15654](https://github.com/hashicorp/consul/issues/15654)]
+* connect: **(Consul Enterprise only)** Implement order-by-locality failover.
+* mesh: Add new permissive mTLS mode that allows sidecar proxies to forward incoming traffic unmodified to the application. This adds `AllowEnablingPermissiveMutualTLS` setting to the mesh config entry and the `MutualTLSMode` setting to proxy-defaults and service-defaults. [[GH-17035](https://github.com/hashicorp/consul/issues/17035)]
+* mesh: Support configuring JWT authentication in Envoy. [[GH-17452](https://github.com/hashicorp/consul/issues/17452)]
+* server: **(Enterprise Only)** added server side RPC requests IP based read/write rate-limiter. [[GH-4633](https://github.com/hashicorp/consul/issues/4633)]
+* server: **(Enterprise Only)** allow automatic license utilization reporting. [[GH-5102](https://github.com/hashicorp/consul/issues/5102)]
+* server: added server side RPC requests global read/write rate-limiter. [[GH-16292](https://github.com/hashicorp/consul/issues/16292)]
+* xds: Add `property-override` built-in Envoy extension that directly patches Envoy resources. [[GH-17487](https://github.com/hashicorp/consul/issues/17487)]
+* xds: Add a built-in Envoy extension that inserts External Authorization (ext_authz) network and HTTP filters. [[GH-17495](https://github.com/hashicorp/consul/issues/17495)]
+* xds: Add a built-in Envoy extension that inserts Wasm HTTP filters. [[GH-16877](https://github.com/hashicorp/consul/issues/16877)]
+* xds: Add a built-in Envoy extension that inserts Wasm network filters. [[GH-17505](https://github.com/hashicorp/consul/issues/17505)]
+
+IMPROVEMENTS:
+
+* * api: Support filtering for config entries. [[GH-17183](https://github.com/hashicorp/consul/issues/17183)]
+* * cli: Add `-filter` option to `consul config list` for filtering config entries. [[GH-17183](https://github.com/hashicorp/consul/issues/17183)]
+* api: Enable setting query options on agent force-leave endpoint. [[GH-15987](https://github.com/hashicorp/consul/issues/15987)]
+* audit-logging: (Enterprise only) enable error response and request body logging [[GH-5669](https://github.com/hashicorp/consul/issues/5669)]
+* audit-logging: **(Enterprise only)** enable error response and request body logging
+* ca: automatically set up Vault's auto-tidy setting for tidy_expired_issuers when using Vault as a CA provider. [[GH-17138](https://github.com/hashicorp/consul/issues/17138)]
+* ca: support Vault agent auto-auth config for Vault CA provider using AliCloud authentication. [[GH-16224](https://github.com/hashicorp/consul/issues/16224)]
+* ca: support Vault agent auto-auth config for Vault CA provider using AppRole authentication. [[GH-16259](https://github.com/hashicorp/consul/issues/16259)]
+* ca: support Vault agent auto-auth config for Vault CA provider using Azure MSI authentication. [[GH-16298](https://github.com/hashicorp/consul/issues/16298)]
+* ca: support Vault agent auto-auth config for Vault CA provider using JWT authentication. [[GH-16266](https://github.com/hashicorp/consul/issues/16266)]
+* ca: support Vault agent auto-auth config for Vault CA provider using Kubernetes authentication. [[GH-16262](https://github.com/hashicorp/consul/issues/16262)]
+* command: Adds ACL enabled to status output on agent startup. [[GH-17086](https://github.com/hashicorp/consul/issues/17086)]
+* command: Allow creating ACL Token TTL with greater than 24 hours with the -expires-ttl flag. [[GH-17066](https://github.com/hashicorp/consul/issues/17066)]
+* connect: **(Enterprise Only)** Add support for specifying "Partition" and "Namespace" in Prepared Queries failover rules.
+* connect: update supported envoy versions to 1.23.10, 1.24.8, 1.25.7, 1.26.2 [[GH-17546](https://github.com/hashicorp/consul/issues/17546)]
+* connect: update supported envoy versions to 1.23.8, 1.24.6, 1.25.4, 1.26.0 [[GH-5200](https://github.com/hashicorp/consul/issues/5200)]
+* fix metric names in /docs/agent/telemetry [[GH-17577](https://github.com/hashicorp/consul/issues/17577)]
+* gateway: Change status condition reason for invalid certificate on a listener from "Accepted" to "ResolvedRefs". [[GH-17115](https://github.com/hashicorp/consul/issues/17115)]
+* http: accept query parameters `datacenter`, `ap` (enterprise-only), and `namespace` (enterprise-only). Both short-hand and long-hand forms of these query params are now supported via the HTTP API (dc/datacenter, ap/partition, ns/namespace). [[GH-17525](https://github.com/hashicorp/consul/issues/17525)]
+* systemd: set service type to notify. [[GH-16845](https://github.com/hashicorp/consul/issues/16845)]
+* ui: Update alerts to Hds::Alert component [[GH-16412](https://github.com/hashicorp/consul/issues/16412)]
+* ui: Update to use Hds::Toast component to show notifications [[GH-16519](https://github.com/hashicorp/consul/issues/16519)]
+* ui: update from <button> and <a> to design-system-components button <Hds::Button> [[GH-16251](https://github.com/hashicorp/consul/issues/16251)]
+* ui: update typography to styles from hds [[GH-16577](https://github.com/hashicorp/consul/issues/16577)]
+
+BUG FIXES:
+
+* Fix a race condition where an event is published before the data associated is commited to memdb. [[GH-16871](https://github.com/hashicorp/consul/issues/16871)]
+* gateways: **(Enterprise only)** Fixed a bug in API gateways where gateway configuration objects in non-default partitions did not reconcile properly. [[GH-17581](https://github.com/hashicorp/consul/issues/17581)]
+* gateways: Fixed a bug in API gateways where binding a route that only targets a service imported from a peer results
+in the programmed gateway having no routes. [[GH-17609](https://github.com/hashicorp/consul/issues/17609)]
+* gateways: Fixed a bug where API gateways were not being taken into account in determining xDS rate limits. [[GH-17631](https://github.com/hashicorp/consul/issues/17631)]
+* peering: Fixes a bug where the importing partition was not added to peered failover targets, which causes issues when the importing partition is a non-default partition. [[GH-16673](https://github.com/hashicorp/consul/issues/16673)]
+* ui: fixes ui tests run on CI [[GH-16428](https://github.com/hashicorp/consul/issues/16428)]
+* xds: Fixed a bug where modifying ACLs on a token being actively used for an xDS connection caused all xDS updates to fail. [[GH-17566](https://github.com/hashicorp/consul/issues/17566)]
+
 ## 1.15.3 (June 1, 2023)
 
 BREAKING CHANGES:

diff --git a/agent/envoyextensions/builtin/http/localratelimit/ratelimit.go b/agent/envoyextensions/builtin/http/localratelimit/ratelimit.go
@@ -60,6 +60,9 @@ func (r *ratelimit) fromArguments(args map[string]interface{}) error {
 	if err := mapstructure.Decode(args, r); err != nil {
 		return fmt.Errorf("error decoding extension arguments: %v", err)
 	}
+	if r.ProxyType == "" {
+		r.ProxyType = string(api.ServiceKindConnectProxy)
+	}
 	return r.validate()
 }
 
@@ -188,7 +191,7 @@ func (r ratelimit) PatchFilter(p extensioncommon.FilterPayload) (*envoy_listener
 }
 
 func validateProxyType(t string) error {
-	if t != "connect-proxy" {
+	if t != string(api.ServiceKindConnectProxy) {
 		return fmt.Errorf("unexpected ProxyType %q", t)
 	}
 

diff --git a/agent/envoyextensions/builtin/http/localratelimit/ratelimit_test.go b/agent/envoyextensions/builtin/http/localratelimit/ratelimit_test.go
@@ -111,6 +111,30 @@ func TestConstructor(t *testing.T) {
 			expectedErrMsg: "cannot parse 'FilterEnforced', -1 overflows uint",
 			ok:             false,
 		},
+		"invalid proxy type": {
+			arguments: makeArguments(map[string]interface{}{
+				"ProxyType":     "invalid",
+				"FillInterval":  30,
+				"MaxTokens":     20,
+				"TokensPerFill": 5,
+			}),
+			expectedErrMsg: `unexpected ProxyType "invalid"`,
+			ok:             false,
+		},
+		"default proxy type": {
+			arguments: makeArguments(map[string]interface{}{
+				"FillInterval":  30,
+				"MaxTokens":     20,
+				"TokensPerFill": 5,
+			}),
+			expected: ratelimit{
+				ProxyType:     "connect-proxy",
+				MaxTokens:     intPointer(20),
+				FillInterval:  intPointer(30),
+				TokensPerFill: intPointer(5),
+			},
+			ok: true,
+		},
 		"valid everything": {
 			arguments: makeArguments(map[string]interface{}{
 				"ProxyType":     "connect-proxy",

diff --git a/agent/envoyextensions/builtin/lua/lua.go b/agent/envoyextensions/builtin/lua/lua.go
@@ -45,6 +45,9 @@ func (l *lua) fromArguments(args map[string]interface{}) error {
 	if err := mapstructure.Decode(args, l); err != nil {
 		return fmt.Errorf("error decoding extension arguments: %v", err)
 	}
+	if l.ProxyType == "" {
+		l.ProxyType = string(api.ServiceKindConnectProxy)
+	}
 	return l.validate()
 }
 
@@ -53,7 +56,7 @@ func (l *lua) validate() error {
 	if l.Script == "" {
 		resultErr = multierror.Append(resultErr, fmt.Errorf("missing Script value"))
 	}
-	if l.ProxyType != "connect-proxy" {
+	if l.ProxyType != string(api.ServiceKindConnectProxy) {
 		resultErr = multierror.Append(resultErr, fmt.Errorf("unexpected ProxyType %q", l.ProxyType))
 	}
 	if l.Listener != "inbound" && l.Listener != "outbound" {

diff --git a/agent/envoyextensions/builtin/lua/lua_test.go b/agent/envoyextensions/builtin/lua/lua_test.go
@@ -54,6 +54,15 @@ func TestConstructor(t *testing.T) {
 			arguments: makeArguments(map[string]interface{}{"Listener": "invalid"}),
 			ok:        false,
 		},
+		"default proxy type": {
+			arguments: makeArguments(map[string]interface{}{"ProxyType": ""}),
+			expected: lua{
+				ProxyType: "connect-proxy",
+				Listener:  "inbound",
+				Script:    "lua-script",
+			},
+			ok: true,
+		},
 		"valid everything": {
 			arguments: makeArguments(map[string]interface{}{}),
 			expected: lua{

diff --git a/agent/envoyextensions/builtin/property-override/property_override.go b/agent/envoyextensions/builtin/property-override/property_override.go
@@ -261,6 +261,9 @@ func (p *propertyOverride) validate() error {
 		}
 	}
 
+	if p.ProxyType == "" {
+		p.ProxyType = api.ServiceKindConnectProxy
+	}
 	if err := validProxyTypes.CheckRequired(string(p.ProxyType), "ProxyType"); err != nil {
 		resultErr = multierror.Append(resultErr, err)
 	}

diff --git a/agent/envoyextensions/builtin/property-override/property_override_test.go b/agent/envoyextensions/builtin/property-override/property_override_test.go
@@ -236,7 +236,7 @@ func TestConstructor(t *testing.T) {
 		// enforces expected behavior until we do. Multi-member slices should be unaffected
 		// by WeakDecode as it is a more-permissive version of the default behavior.
 		"single value Patches decoded as map construction succeeds": {
-			arguments: makeArguments(map[string]any{"Patches": makePatch(map[string]any{})}),
+			arguments: makeArguments(map[string]any{"Patches": makePatch(map[string]any{}), "ProxyType": nil}),
 			expected:  validTestCase(OpAdd, extensioncommon.TrafficDirectionOutbound, ResourceTypeRoute).expected,
 			ok:        true,
 		},

diff --git a/go.mod b/go.mod
@@ -38,11 +38,11 @@ require (
 	github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
 	github.com/hashicorp/consul-awsauth v0.0.0-20220713182709-05ac1c5c2706
 	github.com/hashicorp/consul-net-rpc v0.0.0-20221205195236-156cfab66a69
-	github.com/hashicorp/consul/api v1.20.0
-	github.com/hashicorp/consul/envoyextensions v0.1.2
-	github.com/hashicorp/consul/proto-public v0.2.1
-	github.com/hashicorp/consul/sdk v0.13.1
-	github.com/hashicorp/consul/troubleshoot v0.1.2
+	github.com/hashicorp/consul/api v1.22.0-rc1
+	github.com/hashicorp/consul/envoyextensions v0.3.0-rc1
+	github.com/hashicorp/consul/proto-public v0.4.0-rc1
+	github.com/hashicorp/consul/sdk v0.14.0-rc1
+	github.com/hashicorp/consul/troubleshoot v0.3.0-rc1
 	github.com/hashicorp/go-bexpr v0.1.2
 	github.com/hashicorp/go-checkpoint v0.5.0
 	github.com/hashicorp/go-cleanhttp v0.5.2

diff --git a/website/content/docs/agent/telemetry.mdx b/website/content/docs/agent/telemetry.mdx
@@ -480,10 +480,8 @@ These metrics are used to monitor the health of the Consul servers.
 | `consul.raft.leader.dispatchNumLogs`                | Measures the number of logs committed to disk in a batch.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | logs                              | gauge   |
 | `consul.raft.logstore.verifier.checkpoints_written`     | Counts the number of checkpoint entries written to the LogStore. | checkpoints | counter |
 | `consul.raft.logstore.verifier.dropped_reports`         | Counts how many times the verifier routine was still busy when the next checksum came in and so verification for a range was skipped. If you see this happen, consider increasing the interval between checkpoints with [`raft_logstore.verification.interval`](/consul/docs/agent/config/config-files#raft_logstore_verification) | reports dropped | counter |
-| `consul.raft.logstore.verifier.ranges_verified`         | Counts the number of log ranges for which a verification report has been completed. Refer to [Monitor Raft metrics and logs for WAL
-](/consul/docs/agent/wal-logstore/monitoring) for more information. | log ranges verifications | counter |
-| `consul.raft.logstore.verifier.read_checksum_failures`  | Counts the number of times a range of logs between two check points contained at least one disk corruption. Refer to [Monitor Raft metrics and logs for WAL
-](/consul/docs/agent/wal-logstore/monitoring) for more information. | disk corruptions | counter |
+| `consul.raft.logstore.verifier.ranges_verified`         | Counts the number of log ranges for which a verification report has been completed. Refer to [Monitor Raft metrics and logs for WAL](/consul/docs/agent/wal-logstore/monitoring) for more information. | log ranges verifications | counter |
+| `consul.raft.logstore.verifier.read_checksum_failures`  | Counts the number of times a range of logs between two check points contained at least one disk corruption. Refer to [Monitor Raft metrics and logs for WAL](/consul/docs/agent/wal-logstore/monitoring) for more information. | disk corruptions | counter |
 | `consul.raft.logstore.verifier.write_checksum_failures` | Counts the number of times a follower has a different checksum to the leader at the point where it writes to the log. This could be caused by either a disk-corruption on the leader (unlikely) or some other corruption of the log entries in-flight. | in-flight corruptions | counter |
 | `consul.raft.leader.lastContact`                    | Measures the time since the leader was last able to contact the follower nodes when checking its leader lease. It can be used as a measure for how stable the Raft timing is and how close the leader is to timing out its lease.The lease timeout is 500 ms times the [`raft_multiplier` configuration](/consul/docs/agent/config/config-files#raft_multiplier), so this telemetry value should not be getting close to that configured value, otherwise the Raft timing is marginal and might need to be tuned, or more powerful servers might be needed. See the [Server Performance](/consul/docs/install/performance) guide for more details.                                                                                                               | ms                                | timer   |
 | `consul.raft.leader.oldestLogAge`                   | The number of milliseconds since the _oldest_ log in the leader's log store was written. This can be important for replication health where write rate is high and the snapshot is large as followers may be unable to recover from a restart if restoring takes longer than the minimum value for the current leader. Compare this with `consul.raft.fsm.lastRestoreDuration` and `consul.raft.rpc.installSnapshot` to monitor. In normal usage this gauge value will grow linearly over time until a snapshot completes on the leader and the log is truncated. Note: this metric won't be emitted until the leader writes a snapshot. After an upgrade to Consul 1.10.0 it won't be emitted until the oldest log was written after the upgrade. | ms                                | gauge   |

diff --git a/website/content/docs/api-gateway/configuration/gateway.mdx b/website/content/docs/api-gateway/configuration/gateway.mdx
@@ -58,7 +58,7 @@ The following outline shows how to format the configurations in the `Gateway` ob
 This topic provides details about the configuration parameters.
 
 ### gatewayClassName
-Specifies the name of the [`GatewayClass`](/consul/docs/api-gateway/configuration/gatewayclass) resource used for the `Gateway` instance. Unless you are using a custom [GatewayClass](/consul/docs/api-gateway/configuration/gatewayclass), this value should be set to `consul-api-gateway`.
+Specifies the name of the [`GatewayClass`](/consul/docs/api-gateway/configuration/gatewayclass) resource used for the `Gateway` instance. Unless you are using a custom [GatewayClass](/consul/docs/api-gateway/configuration/gatewayclass), this value should be set to `consul`.
 * Type: string
 * Required: required