The `-type` option for `volume status` is a UX papercut because for many
clusters there will be only one sort of volume in use. Update the CLI so that
the default behavior is to query CSI and/or DHV.
This behavior is subtly different when the user provides an ID or not. If the
user doesn't provide an ID, we query both CSI and DHV and show both tables. If
the user provides an ID, we query DHV first and then CSI, and show only the
appropriate volume. Because DHV IDs are UUIDs, we're sure we won't have
collisions between the two. We only show errors if both queries return an error.
Fixes: https://hashicorp.atlassian.net/browse/NET-12214
Our vocabulary around scheduler behaviors outside of the `reschedule` and
`migrate` blocks leaves room for confusion around whether the reschedule tracker
should be propagated between allocations. There are effectively five different
behaviors we need to cover:
* restart: when the tasks of an allocation fail and we try to restart the tasks
in place.
* reschedule: when the `restart` block runs out of attempts (or the allocation
fails before tasks even start), and we need to move
the allocation to another node to try again.
* migrate: when the user has asked to drain a node and we need to move the
allocations. These are not failures, so we don't want to propagate the
reschedule tracker.
* replacement: when a node is lost, we don't count that against the `reschedule`
tracker for the allocations on the node (it's not the allocation's "fault",
after all). We don't want to run the `migrate` machinery here here either, as we
can't contact the down node. To the scheduler, this is effectively the same as
if we bumped the `group.count`
* replacement for `disconnect.replace = true`: this is a replacement, but the
replacement is intended to be temporary, so we propagate the reschedule tracker.
Add a section to the `reschedule`, `migrate`, and `disconnect` blocks explaining
when each item applies. Update the use of the word "reschedule" in several
places where "replacement" is correct, and vice-versa.
Fixes: https://github.com/hashicorp/nomad/issues/24918
Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>
Similarly to #6732 it removes checking affinity and spread for inplace update.
Both affinity and spread should be as soft preference for Nomad scheduler rather than strict constraint. Therefore modifying them should not trigger job reallocation.
Fixes#25070
Co-authored-by: Tim Gross <tgross@hashicorp.com>
* Add dead (stopped) to status mapping to clarify Stopped
CE-816
* Pull status mapping into partial and include in job status command
* change `complete` to dead in table after discuss with Michael
* added clarifications; add CLI status definitions
* fixed line endings
* fixed typoce816dead
* Docs SEO: task drivers and plugins; refactor virt section
* add redirects for virt driver files
* Some updates. committing rather than stashing
* fix content-check errors
* Remove docs/devices/ and redirect to plugins/devices
* Update docs/drivers descriptions
* Move USB device plugin up a level. Finish descriptions.
* Apply suggestions from Jeff's code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply title case suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* apply title case suggestions; fix indentation
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Internally, sizes are always in binary units; this documentation is misleading and implies that they work in decimal units.
Without going through and replacing _every_ "MB" -> "MiB" this is the best way to hint to developers that binary sizes are used.
* Adds Actions to job status command output
* Adds Actions to job status command output
* Status documentation updated to show actions and formatJobActions no longer cares about pipe delineation
We have a document describing the various approaches to storage that surveys the
landscape and makes recommendations based on the user's environment. Add dynamic
host volumes to this document.
Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>
In Nomad 1.10, quotas will use the new `storage` block to specify limits on host
volume and variables storage. Previous PRs have updated the upgrade guide noting
the deprecation of the existing `variables_limit` field.
Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
Dynamic host volumes use the same specification file as CSI volumes but require
a different set of parameters and have different semantics. This changeset
splits the volume specification page into separate CSI and dynamic host
volumes spec pages.
While migrating the CSI page, I've also edited it to bring it more in line with
the style guide: removed passive voice and future tense, inclusive language,
alphabetized the (chaotic!) parameters list, etc.
Ref: https://github.com/hashicorp/nomad/pull/24797
Ref: https://hashicorp.atlassian.net/browse/NET-11482
Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>
In anticipation of having quotas for dynamic host volumes, we want the user
experience of the storage limits to feel integrated with the other resource
limits. This is currently prevented by reusing the `Resources` type instead of
having a specific type for `QuotaResources`.
Update the quota limit/usage types to use a `QuotaResources` that includes a new
storage resources quota block. The wire format for the two types are compatible
such that we can migrate the existing variables limit in the FSM.
Also fixes improper parallelism in the quota init test where we change working
directory to avoid file write conflicts but this breaks when multiple tests are
executed in the same process.
Ref: https://github.com/hashicorp/nomad-enterprise/pull/2096
* Update who-uses-nomad.mdx
Our new contract with Roblox states that we can't mention anywhere on our sites that they use us.
* Update who-uses-nomad.mdx
Edited the sentence above the companies list to more accurately reflect them.
Also added Target to the list with a link to their case study.
When the Nomad client restarts and restores allocations, the network namespace
for an allocation may exist but no longer be correctly configured. For example,
if the host is rebooted and the task was a Docker task using a pause container,
the network namespace may be recreated by the docker daemon.
When we restore an allocation, use the CNI "check" command to verify that any
existing network namespace matches the expected configuration. This requires CNI
plugins of at least version 1.2.0 to avoid a bug in older plugin versions that
would cause the check to fail.
If the check fails, destroy the network namespace and try to recreate it from
scratch once. If that fails in the second pass, fail the restore so that the
allocation can be recreated (rather than silently having networking fail).
This should fix the gap left #24650 for Docker task drivers and any other
drivers with the `MustInitiateNetwork` capability.
Fixes: https://github.com/hashicorp/nomad/issues/24292
Ref: https://github.com/hashicorp/nomad/pull/24650
Adds an additional check in the Keyring.Delete RPC to make sure we're not
trying to delete a key that's been used to encrypt a variable. It also adds a
-force flag for the CLI/API to sidestep that check.
The Nomad client can now optionally emit telemetry data from the
prerun and prestart hooks. This allows operators to monitor and
alert on failures and time taken to complete.
The new datapoints are:
- nomad.client.alloc_hook.prerun.success (counter)
- nomad.client.alloc_hook.prerun.failed (counter)
- nomad.client.alloc_hook.prerun.elapsed (sample)
- nomad.client.task_hook.prestart.success (counter)
- nomad.client.task_hook.prestart.failed (counter)
- nomad.client.task_hook.prestart.elapsed (sample)
The hook execution time is useful to Nomad engineering and will
help optimize code where possible and understand job specification
impacts on hook performance.
Currently only the PreRun and PreStart hooks have telemetry
enabled, so we limit the number of new metrics being produced.
