Docs: 1.10 release notes, some factoring, sentinel apply update (#25433)

* Docs: 1.10 release notes and upgrade factoring

* Update based on code review suggestions

* add CLI for disabling UI URL hints

* fix indentation

* nav: list release notes in reverse order

fix broken link to v1.6.x docs

* Update PKCE section from Daniel's latest PR

* update pkce per daniel's suggestion

* Add dynamic host volumes governance section from blog
This commit is contained in:
Aimee Ukasick
2025-04-09 17:43:58 -05:00
committed by GitHub
parent 95c914624e
commit 87aabc9af2
13 changed files with 326 additions and 85 deletions

View File

@@ -22,6 +22,9 @@ The `sentinel apply` command requires two arguments, the policy name and the
policy file. The policy file can be read from stdin by specifying "-" as the
file name.
Additionally, you must specify the `-scope` option. Refer to the [`-scope` field
description](/nomad/docs/commands/sentinel/apply#scope) for more information.
Sentinel commands are only available when ACLs are enabled. This command
requires a management token.
@@ -33,8 +36,14 @@ requires a management token.
- `-description` : Sets a human readable description for the policy
- `-scope` : (default: submit-job) Sets the scope of the policy and when it
should be enforced.
- `-scope` : Required. Sets the scope of the policy and when it
should be enforced. Specify one of the following:
- The `submit-job` scope for registering jobs
- The `submit-host-volume` scope for creating or updating dynamic host
volumes.
Refer to the [Sentinel guide](/nomad/docs/enterprise/sentinel) for scope details.
- `-level` : (default: advisory) Sets the enforcement level of the policy. Must
be one of advisory, soft-mandatory, hard-mandatory.
@@ -44,6 +53,6 @@ requires a management token.
Write a policy:
```shell-session
$ nomad sentinel write -description "My test policy" foo test.sentinel
$ nomad sentinel write -scope "submit-job" -description "My test policy" foo test.sentinel
Successfully wrote "foo" Sentinel policy!
```

View File

@@ -0,0 +1,220 @@
---
layout: docs
page_title: Nomad v1.10.x release notes
description: >-
HashiCorp Nomad version 1.10.x release notes. New features include dynamic host volumes, OIDC Private Key JWT (client assertions), and OIDC Proof Key for Code Exchange (PKCE). Updates to the Container Storage Interface (CSI) volume delete and volume status commands. CSI volume and plugin events added to the event stream. Added UI URL hints to the end of common CLI commands and a -ui flag to auto-open them in a browser.
---
# Nomad 1.10.x release notes
We are pleased to announce the following Nomad updates.
## Release highlights
### Dynamic host volumes
The dynamic host volumes feature brings a persistent storage option to your
workload allocations.
Nomad dynamic host volumes manage storage for stateful workloads without
requiring a restart of the Nomad nodes to apply configuration changes. You
create dynamic host volumes via the CLI or API and then configure the job with
the [`volume`](/nomad/docs/job-specification/volume) and
[`volume_mount`](/nomad/docs/job-specification/volume_mount) blocks in the job
specification.
Host volumes mount paths from the Nomad client into allocations. Nomad is aware
of host volume availability and makes use of it for job scheduling. However,
Nomad does not know about the volume's underlying characteristics, so you can
use host volumes for both local persistent storage and for highly
available networked storage.
#### Dynamic host volume governance <EnterpriseAlert inline product="nomad"/>
Providing guardrails to platform consumers is an important aspect of the storage
provisioning workflow when leveraging host volumes across a shared Nomad
cluster. Nomad Enterprise supports these new capabilities to provide
governance when provisioning host volumes:
- **Sentinel dynamic host volume objects**
During volume creation, Nomad can evaluate all of the details within the
dynamic host volume specification against Sentinel policies
that define and enforce specific patterns.
For example, a policy that enforces the storage tier based on the environment
or namespace specified would allow reserving more expensive NVME storage for
specific workloads. Being able to apply policy to the volume specification
gives you a method to enforce specific patterns while providing platform
consumers with more flexibility around self-service volume provisioning. Refer
to the [Sentinel policy reference](/nomad/docs/enterprise/sentinel#sentinel-dynamic-host-volume-objects) for more information.
- **Resource quota support**
Nomads resource quota system now includes coverage for host volume capacity
limits that you can apply to provisioned storage within a specific namespace.
This helps control storage consumption within a namespace based on the maximum
capacities defined during creation or when making updates to the maximum
capacities over the lifecycle of the volume. Refer to the [Resource quota
specification](/nomad/docs/other-specifications/quota) for more information.
- **Namespace and node pool validation**:
Dynamic host volumes live within the context of a specific namespace when
created. When Nomad provisions volumes in a namespace targeting a specific
node pool, Nomad evaluates the namespace node pool configuration to ensure
that volume creation aligns with job placement rules for node pools. Refer to
the [Namespace specification](/nomad/docs/other-specifications/namespace) for
details on `node_pool_config` parameters.
#### Resources
Refer to the following resources to learn more about dynamic host volumes:
- [Host volumes section](/nomad/docs/operations/stateful-workloads#host-volumes)
in the _Considerations for stateful workloads_ guide for an overview and
comparison of storage options
- [Host volumes plugin
specification](/nomad/docs/concepts/plugins/storage/host-volumes) for examples
of how to write your own plugin to dynamically configure persistent storage on
your Nomad client nodes
- [Use Nomad dynamic host volumes to persist data for stateful workloads
tutorial](/nomad/tutorials/stateful-workloads/stateful-workloads-dynamic-host-volumes)
to learn how to create and use a dynamic host volume for persistent storage
### OpenID Connect (OIDC) enhancements
Nomad 1.10 extends Nomad's OIDC SSO login feature with [Private Key JWT][pkjwt]
and [Proof Key for Code Exchange (PKCE)][pkce].
#### Private Key JWT
Private Key JWT, also called client assertions, is a more secure alternative for
client secrets. Instead of sending a simple secret, Nomad builds a JWT and signs
it with a value that the OIDC provider verifies. In this approach, Nomad asserts
a valid OIDC client without sending any secret information over the network.
#### Proof Key for Code Exchange (PKCE)
PKCE adds an extra layer of security to any OIDC auth method for both client
secrets and client assertions.
Set the [ACL auth method `OIDCEnablePKCE`
parameter](/nomad/api-docs/acl/auth-methods#oidcenablepkce) to `true` to turn
on this extra security.
Note that not all OIDC providers support PKCE. In addition to enabling PKCE in
Nomad, you may need to enable it in your OIDC provider's configuration.
#### Resources
- [OIDC auth method guide][oidc-concepts] for details on using OIDC with Nomad
- [OIDC troubleshooting guide][oidc-trouble] to review common issues and tips
for setting up OIDC
- [Authenticate users with SSO and Keycloak tutorial][oidc-tutorial] to
configure Nomad and the Keycloak identity provider to automatically grant
permissions in Nomad ACL.
### Container Storage Interface (CSI) enhancements
We added the following:
- CSI volume and plugin events to the event stream
- Volume capabilities to the `nomad volume status` command output
- The ability to use a volume ID prefix search and wildcard namespace with the
[`nomad volume delete` command](/nomad/docs/commands/volume/delete). Refer to
the [GitHub pull request](https://github.com/hashicorp/nomad/pull/24997) for
details. Example usage:
```shell-session
$ nomad volume create ./internal-plugin.volume.hcl
==> Created host volume internal-plugin with ID aeea91a0-06df-c16e-5403-ff82a2f28fd4
✓ Host volume "aeea91a0" ready
2025-01-31T15:55:14-05:00
ID = aeea91a0-06df-c16e-5403-ff82a2f28fd4
Name = internal-plugin
Namespace = default
Plugin ID = mkdir
Node ID = b4611abd-d4a8-c83a-b05e-7d9f5b44a179
Node Pool = default
Capacity = 0 B
State = ready
Host Path = /run/nomad/dev/data/host_volumes/aeea91a0-06df-c16e-5403-ff82a2f28fd4
$ nomad volume delete -type host aeea91a0
Successfully deleted volume "aeea91a0-06df-c16e-5403-ff82a2f28fd4"!
```
### UI URL hints added to CLI commands
We added UI URL hints to the end of common CLI commands and a `-ui` flag to
automatically open the generated link in your browser.
Showing UI URL hints is enabled by default. You have two options for turning off
this feature:
- Server: Modify the [`show_cli_hints`
parameter](/nomad/docs/configuration/ui#show_cli_hints) in your agent's `ui`
block configuration.
- CLI: Set the `NOMAD_CLI_SHOW_HINTS` environment variable to `0` or `false`.
```shell-session
$ nomad status
No running jobs
==> View and manage Nomad jobs in the Web UI: https://localhost:4646/ui/jobs
$ export NOMAD_CLI_SHOW_HINTS=0
$ nomad status
No running jobs
```
## Breaking changes
@include 'release-notes/v1-10/breaking-go-sdk.mdx'
@include 'release-notes/v1-10/breaking-plugin-dir.mdx'
@include 'release-notes/v1-10/breaking-vault-consul-token.mdx'
@include 'release-notes/v1-10/breaking-consul-template.mdx'
@include 'release-notes/v1-10/breaking-disconnect-fields-removed.mdx'
@include 'release-notes/v1-10/breaking-remove-remote-task-driver-support.mdx'
@include 'release-notes/v1-10/breaking-sentinel-apply.mdx'
@include 'release-notes/v1-10/breaking-affinity-spread.mdx'
## Deprecations
@include 'release-notes/v1-10/deprecate-variable-limits.mdx'
## Upgrade details
For more detailed information, refer to the [upgrade details
page][upgrade] and the [GitHub releases changelogs][github-releases].
## Known issues
None.
## Changelogs
These links take you to the changelogs on the GitHub website.
- [v1.10.0-beta.1](https://github.com/hashicorp/nomad/releases/tag/v1.10.0-beta.1)
[consul-integration]: /nomad/docs/integrations/consul/acl
[vault-integration]: /nomad/docs/integrations/vault/acl
[GH-18529]: https://github.com/hashicorp/nomad/issues/18529
[upgrade]: /nomad/docs/upgrade/upgrade-specific#nomad-1-10-0
[github-releases]: https://github.com/hashicorp/nomad/releases/
[pkjwt]: https://oauth.net/private-key-jwt/
[pkce]: https://oauth.net/2/pkce/
[oidc-concepts]: /nomad/docs/concepts/acl/auth-methods/oidc#client-assertions
[oidc-trouble]: /nomad/docs/concepts/acl/auth-methods/oidc#oidc-configuration-troubleshooting
[oidc-tutorial]: /nomad/tutorials/single-sign-on/sso-oidc-keycloak

View File

@@ -1,11 +1,11 @@
---
layout: docs
page_title: Upgrade Guides
page_title: Nomad version-specific upgrade guides
description: |-
Review upgrade details for specific versions of Nomad.
---
# Upgrade Guides
# Nomad version-specific upgrade guides
The [upgrading page](/nomad/docs/upgrade) covers the details of doing a standard
upgrade. However, specific versions of Nomad may have more details provided for
@@ -14,90 +14,21 @@ used to document those details separately from the standard upgrade flow.
## Nomad 1.10.0
#### Quota specification variable_limits deprecated
@include 'release-notes/v1-10/deprecate-variable-limits.mdx'
In Nomad 1.10.0, the quota specification's `variable_limits` field is
deprecated. It is replaced by a new `storage` block with a `variables` field,
under the `region_limit` block. Existing quotas will be automatically migrated
during server upgrade. The `variables_limit` field will be removed from the
quota specification in Nomad 1.12.0.
@include 'release-notes/v1-10/breaking-disconnect-fields-removed.mdx'
#### Deprecated Disconnect fields removed
@include 'release-notes/v1-10/breaking-go-sdk.mdx'
In Nomad 1.8, the `disconnect` block was introduced to replace the `max_client_disconnect`
`stop_after_client_disconnect`, and `prevent_reschedule_on_list` fields.
In Nomad 1.10 these fields have been removed and will be ignored if specified. Jobs
should migrate to using the `disconnect` block prior to upgrading.
@include 'release-notes/v1-10/breaking-remove-remote-task-driver-support.mdx'
#### Go SDK API change for quota limits
@include 'release-notes/v1-10/breaking-plugin-dir.mdx'
In Nomad 1.10.0, the Go API for quotas has a breaking change. The
`QuotaSpec.RegionLimit` field is now of type `QuotaResources` instead of
`Resources`. The `QuotaSpec.VariablesLimit` field is deprecated in lieu of
`QuotaSpec.RegionLimit.Storage.Variables` and will be removed in Nomad 1.12.0.
@include 'release-notes/v1-10/breaking-sentinel-apply.mdx'
#### Remote task driver support removed
@include 'release-notes/v1-10/breaking-affinity-spread.mdx'
All support for remote task driver capabilities has been removed in Nomad 1.10.0.
Drivers with the `RemoteTasks` capability will no longer be detached in the event
the allocation is lost, nor will remote tasks be detached when a node is drained.
Workloads running as remote tasks should be migrated prior to upgrading.
#### Loading Binaries from `plugin_dir` Without Configuration
Plugins stored within the [`plugin_dir`](/nomad/docs/configuration#plugin_dir)
will now only be loaded when they have a corresponding
[`plugin`](/nomad/docs/configuration/plugin) block in the agent configuration
file.
#### Affinity and spread updates are non-destructive
In Nomad 1.10.0, a scheduler bug was fixed so that updates to `affinity` and
`spread` blocks are no longer destructive. After a job update that changes only
these blocks, existing allocations remain running with their job version
incremented. If you were relying on the previous behavior to redistribute
workloads, you can force a destructive update by changing fields that require
one, such as the `meta` block.
#### Vault and Consul integration changes
Nomad 1.10.0 removes the previously deprecated token-based authentication workflow
for Vault and Consul. Nomad clients must now use a task's [Workload Identity](/nomad/docs/concepts/workload-identity)
to authenticate to Vault and Consul and obtain a token specific to the task.
This table lists removed fields and the new workflow.
| Field | Configuration | New Workflow |
| ------ | ------------ | ------------ |
| [`vault.allow_unauthenticated`](/nomad/docs/v1.8.x/configuration/vault#allow_unauthenticated) | Agent | Tasks should use a workload identity. Do not use a Vault token. |
| [`vault.task_token_ttl`]( /nomad/docs/v1.8.x/configuration/vault#task_token_ttl) | Agent | With workload identity, tasks receive their TTL configuration from the Vault role. |
| [`vault.token`](/nomad/docs/v1.8.x/configuration/vault#token) | Agent | Nomad agents use the workload identity when making requests to authenticated endpoints. |
| [`vault.policies`](/nomad/docs/v1.8.x/job-specification/vault#policies) | Job specification | Configure and use a Vault role. |
Before upgrading to Nomad 1.10, perform the following tasks:
1. Configure Vault to work with workload identity.
1. Migrate all workloads to use workload identity.
Refer to [Migrating to Using Workload Identity with Vault](/nomad/docs/v1.8.x/integrations/vault/acl#migrating-to-using-workload-identity-with-vault) for more
details.
#### Consul template implicit workload identity removal
Nomad no longer creates an implicit Consul identity for workloads that don't
register services with Consul. Tasks that require Consul tokens for template
rendering must include a [`consul` block](/nomad/docs/job-specification/consul) or specify an [`identity`](/nomad/docs/job-specification/identity#identity-block)
#### OIDC login with PKCE
Nomad now enables
[Proof Key for Code Exchange (PKCE)](https://oauth.net/2/pkce/)
by default for new OIDC auth methods. Existing auth methods remain unaffected.
Note that even if PKCE is enabled in Nomad, some OIDC providers may require you
to also enable it in their configuration.
Set the [`OIDCEnablePKCE`](/nomad/api-docs/acl/auth-methods#oidcenablepkce)
option to `false` to turn off this extra security.
@include 'release-notes/v1-10/breaking-vault-consul-token.mdx'
## Nomad 1.9.5
@@ -521,7 +452,7 @@ In practice this means that `exec` tasks such as Vault which use the `mlock`
system call will now need to explicitly add the `ipc_lock` capability.
First [allow the `ipc_lock` capability in the Client
configuration](/nomad/docs/dv1.6.x/drivers/exec#allow_caps):
configuration](/nomad/docs/v1.6.x/drivers/exec#allow_caps):
```hcl
plugin "exec" {

View File

@@ -0,0 +1,8 @@
#### Affinity and spread updates are non-destructive
We fixed a scheduler bug so that updates to `affinity` and
`spread` blocks are no longer destructive. After a job update that changes only
these blocks, existing allocations remain running with their job version
incremented. If you were relying on the previous behavior to redistribute
workloads, you can force a destructive update by changing fields that require
one, such as the `meta` block.

View File

@@ -0,0 +1,6 @@
#### Consul template implicit workload identity removal
Nomad no longer creates an implicit Consul identity for workloads that don't
register services with Consul. Tasks that require Consul tokens for template
rendering must include a [`consul` block](/nomad/docs/job-specification/consul)
or specify an [`identity`](/nomad/docs/job-specification/identity#identity-block).

View File

@@ -0,0 +1,7 @@
#### Nomad 1.8 deprecated `disconnect` fields removed
In Nomad 1.8, we introduced the `disconnect` block to replace the
`max_client_disconnect`, `stop_after_client_disconnect`, and
`prevent_reschedule_on_list` fields. In Nomad 1.10, we removed these fields, and
Nomad will ignore them if specified. Jobs should migrate to using the
[`disconnect` block](/nomad/docs/job-specification/disconnect) prior to upgrading.

View File

@@ -0,0 +1,6 @@
#### Go SDK API change for quota limits
In Nomad 1.10.0, the Go API for quotas has a breaking change. The
`QuotaSpec.RegionLimit` field is now of type `QuotaResources` instead of
`Resources`. The `QuotaSpec.VariablesLimit` field is deprecated in lieu of
`QuotaSpec.RegionLimit.Storage.Variables` and will be removed in Nomad 1.12.0.

View File

@@ -0,0 +1,6 @@
#### Loading binaries from `plugin_dir` without configuration
Plugins stored within the [`plugin_dir`](/nomad/docs/configuration#plugin_dir)
will now only be loaded when they have a corresponding
[`plugin`](/nomad/docs/configuration/plugin) block in the agent configuration
file. Nomad now skips any plugin found without a corresponding configuration block.

View File

@@ -0,0 +1,7 @@
#### Remote task driver support removed
In Nomad 1.10.0, we removed all support for remote task driver
capabilities. Nomad no longer detaches drivers with the `RemoteTasks` capability
when an allocation is lost. Also, Nomad does not detach remote tasks
when a node is drained. Workloads running as remote tasks should be migrated
prior to upgrading.

View File

@@ -0,0 +1,5 @@
#### Sentinel apply command requires scope <EnterpriseAlert product="nomad" inline/>
To prevent accidentally adding policies for volumes to the job scope, the [`nomad sentinel apply` command](/nomad/docs/commands/sentinel/apply) now
requires the `-scope` option. Refer to the [GitHub pull
request](https://github.com/hashicorp/nomad/pull/24601) for details.

View File

@@ -0,0 +1,25 @@
#### Vault and Consul integration changes
Nomad 1.10.0 removes the previously deprecated token-based authentication
workflow for Vault and Consul. Nomad clients must now use a task's [workload
identity](/nomad/docs/concepts/workload-identity) to authenticate to Vault and
Consul and obtain a token specific to the task.
This table lists removed Vault fields and the new workflow.
| Field | Configuration | New Workflow |
| ------ | ------------ | ------------ |
| [`vault.allow_unauthenticated`](/nomad/docs/v1.8.x/configuration/vault#allow_unauthenticated) | Agent | Tasks should use a workload identity. Do not use a Vault token. |
| [`vault.task_token_ttl`]( /nomad/docs/v1.8.x/configuration/vault#task_token_ttl) | Agent | With workload identity, tasks receive their TTL configuration from the Vault role. |
| [`vault.token`](/nomad/docs/v1.8.x/configuration/vault#token) | Agent | Nomad agents use the workload identity when making requests to authenticated endpoints. |
| [`vault.policies`](/nomad/docs/v1.8.x/job-specification/vault#policies) | Job specification | Configure and use a Vault role. |
Before upgrading to Nomad 1.10, perform the following tasks:
1. Configure Vault and Consul to work with workload identity.
1. Migrate all workloads to use workload identity.
Refer to the following guides for more information:
- [Migrating to using workload identity with Vault](/nomad/docs/v1.8.x/integrations/vault/acl#migrating-to-using-workload-identity-with-vault)
- [Migrating to using workload identity with Consul](/nomad/docs/v1.8.x/integrations/consul/acl#migrating-to-using-workload-identity-with-consul)

View File

@@ -0,0 +1,7 @@
#### Quota specification variable_limits deprecated <EnterpriseAlert product="nomad" inline/>
The quota specification's `variable_limits` field is deprecated. We replaced it
with a new [`storage` block](/nomad/docs/other-specifications/quota#storage)
with a `variables` field, under the `region_limit` block. Existing quotas will
be automatically migrated during server upgrade. We will remove the
`variables_limit` field from the quota specification in Nomad 1.12.0.

View File

@@ -65,12 +65,16 @@
"path": "release-notes/nomad/upcoming"
},
{
"title": "v1.8.x",
"path": "release-notes/nomad/v1_8_x"
"title": "v1.10.x",
"path": "release-notes/nomad/v1-10-x"
},
{
"title": "v1.9.x",
"path": "release-notes/nomad/v1_9_x"
},
{
"title": "v1.8.x",
"path": "release-notes/nomad/v1_8_x"
}
]
}