Merge pull request #12586 from hashicorp/f-local-si-token

connect: create SI tokens in local scope

.changelog/8068.txt

@@ -0,0 +1,10 @@
+```release-note:breaking-change
+connect: Consul Service Identity ACL tokens automatically generated for Connect services are now
+created as Local rather than Global tokens. Nomad clusters with Connect services making cross-Consul
+datacenter requests will need to ensure their Consul agents are configured with anonymous ACL tokens
+of sufficient node and service read permissions.
+```
+
+```release-note:breaking-change
+connect: The minimum Consul version supported by Nomad's Connect integration is now Consul v1.8.0.
+```

nomad/consul.go

@@ -290,6 +290,7 @@ func (c *consulACLsAPI) CreateToken(ctx context.Context, sir ServiceIdentityRequ
 		Description:       sir.Description(),
 		ServiceIdentities: []*api.ACLServiceIdentity{{ServiceName: service}},
 		Namespace:         sir.ConsulNamespace,
+		Local:             true,
 	}
 
 	// Ensure we are under our rate limit.

nomad/job_endpoint_hook_connect.go

@@ -68,11 +68,11 @@ func connectGatewayDriverConfig(hostNetwork bool) map[string]interface{} {
 // connectSidecarVersionConstraint is used when building the sidecar task to ensure
 // the proper Consul version is used that supports the necessary Connect
 // features. This includes bootstrapping envoy with a unix socket for Consul's
-// gRPC xDS API.
+// gRPC xDS API, and support for generating local service identity tokens.
 func connectSidecarVersionConstraint() *structs.Constraint {
 	return &structs.Constraint{
 		LTarget: "${attr.consul.version}",
-		RTarget: ">= 1.6.0-beta1",
+		RTarget: ">= 1.8.0",
 		Operand: structs.ConstraintSemver,
 	}
 }

website/content/docs/integrations/consul-connect.mdx

@@ -8,9 +8,6 @@ description: >-
 
 # Consul Service Mesh
 
-~> **Note:** This guide requires Nomad 0.10.0 or later and Consul 1.6.0 or
-later.
-
 ~> **Note:** Nomad's service mesh integration requires Linux network namespaces.
 Consul service mesh will not run on Windows or macOS.
 
@@ -58,7 +55,7 @@ The Consul service mesh integration with Nomad requires [Consul 1.6 or
 later.](https://releases.hashicorp.com/consul/1.6.0/) The Consul agent can be
 run in dev mode with the following command:
 
-**Note**: Nomad's Consul service mesh integration requires Consul in your `$PATH`
+~> **Note:** Nomad's Consul service mesh integration requires Consul in your `$PATH`
 
 ```shell-session
 $ consul agent -dev
@@ -96,6 +93,23 @@ For JSON configurations:
 }
 ```
 
+#### Consul ACLs
+
+~> **Note:** Starting in Nomad v1.3.0, Consul Service Identity ACL tokens automatically
+generated by Nomad on behalf of Connect enabled services are now created in [`Local`]
+rather than Global scope, and are no longer replicated globally.
+
+To facilitate cross-Consul datacenter requests of Connect services registered by
+Nomad, Consul agents will need to be configured with [default anonymous][anon_token]
+ACL tokens with ACL policies of sufficient permissions to read service and node
+metadata pertaining to those requests. This mechanism is described in Consul [#7414][consul_acl].
+A typical Consul agent anonymous token may contain an ACL policy such as:
+
+```hcl
+service_prefix "" { policy = "read" }
+node_prefix    "" { policy = "read" }
+```
+
 ### Nomad
 
 Nomad must schedule onto a routable interface in order for the proxies to
@@ -326,6 +340,7 @@ dashes (`-`) are converted to underscores (`_`) in environment variables so
 
 ## Limitations
 
+- The minimum Consul version to use Connect with Nomad is Consul v1.8.0.
 - The `consul` binary must be present in Nomad's `$PATH` to run the Envoy
   proxy sidecar on client nodes.
 - Consul service mesh using network namespaces is only supported on Linux.
@@ -333,4 +348,7 @@ dashes (`-`) are converted to underscores (`_`) in environment variables so
   connections while the Nomad agent is restarting.
 
 [count-dashboard]: /img/count-dashboard.png
+[consul_acl]: https://github.com/hashicorp/consul/issues/7414
 [gh-9907]: https://github.com/hashicorp/nomad/issues/9907
+[`Local`]: https://www.consul.io/docs/security/acl/acl-tokens#token-attributes
+[anon_token]: https://www.consul.io/docs/security/acl/acl-tokens#special-purpose-tokens

website/content/docs/upgrade/upgrade-specific.mdx

@@ -126,6 +126,24 @@ connect {
 }
 ```
 
+#### Changes to Consul Connect Service Identity Tokens
+
+Starting with Nomad 1.3.0, Consul Service Identity Tokens created automatically
+by Nomad on behalf of Connect services will now be created as [`Local`] tokens. These
+tokens will no longer be replicated globally. To facilitate cross-Consul datacenter
+requests of Connect services registered by Nomad, Consul agents will need to be
+configured with [default anonymous][anon_token] ACL tokens with ACL policies of
+sufficient permissions to read service and node metadata pertaining to those
+requests. This mechanism is described in Consul [#7414][consul_acl].
+A typical Consul agent anonymous token may contain an ACL policy such as:
+
+```hcl
+service_prefix "" { policy = "read" }
+node_prefix    "" { policy = "read" }
+```
+
+The minimum version of Consul supported by Nomad's Connect integration is now Consul v1.8.0.
+
 #### Linux Control Groups Version 2
 
 Starting with Nomad 1.3.0, Linux systems configured to use [cgroups v2][cgroups2]
@@ -1376,3 +1394,6 @@ deleted and then Nomad 0.3.0 can be launched.
 [cap_drop_exec]: /docs/drivers/exec#cap_drop
 [`log_file`]: /docs/configuration#log_file
 [Upgrading to Raft Protocol 3]: /docs/upgrade#upgrading-to-raft-protocol-3
+[`Local`]: https://www.consul.io/docs/security/acl/acl-tokens#token-attributes
+[anon_token]: https://www.consul.io/docs/security/acl/acl-tokens#special-purpose-tokens
+[consul_acl]: https://github.com/hashicorp/consul/issues/7414