* Fixes an issue where system jobs' status were set to Scaled Down when their allocs get garbage collected
* Added to aggregateAllocStatus acceptance test and changelog
In #16872 we added support for unix domain sockets, but this required mutating
the `Config` when parsing the address so as to remove the port number. In #23785
we fixed a bug where if the configuration was used across multiple clients that
mutation would happen multiple times and the address would be incorrectly
parsed.
When making `alloc log`, `alloc fs`, or `alloc exec` calls where we have
line-of-sight to the client, we attempt to make a HTTP API call directly to the
client node. So we create a new API client from the same configuration and then
set the address. But in this case we copy the private `url` field and that
causes the URL parsing to be skipped for the new client.
This results in the region always being set to the string literal
`"global"` (because of mTLS handling code introduced all the way back in
4d3b75d867), unless the user has set the region specifically. This fails with
an error "no path to region" when the cluster isn't non-global and requests are
sent to a non-leader.
Arguably the "right" way of fixing this would be for `ClientConfig` not to
change the API client's region to `"global"` in the first place, but as this is
a public API and extremely longstanding behavior, it could potentially be a
breaking change for some downstream consumers. Instead, we'll avoid copying the
private `url` field so that the new address is re-parsed.
Fixes: https://github.com/hashicorp/nomad/issues/24635
Fixes: https://github.com/hashicorp/nomad/issues/24609
Ref: https://github.com/hashicorp/nomad/pull/16872
Ref: https://github.com/hashicorp/nomad/pull/23785
Ref: 4d3b75d867
When a Nomad host reboots, the network namespace files in the tmpfs in
`/var/run` are wiped out. So when we restore allocations after a host reboot, we
need to be able to restore both the network namespace and the network
configuration. But because the netns is newly created and we need to run the CNI
plugins again, this create potential conflicts with the IPAM plugin which has
written state to persistent disk at `/var/lib/cni`. These IPs aren't the ones
advertised to Consul, so there's no particular reason to keep them around after
a host reboot because all virtual interfaces need to be recreated too.
Reconfigure the CNI bridge configuration to use `/var/run/cni` as its state
directory. We already expect this location to be created by CNI because the
netns files are hard-coded to be created there too in `libcni`.
Note this does not fix the problem described for Docker in #24292 because that
appears to be related to the netns itself being restored unexpectedly from
Docker's state.
Ref: https://github.com/hashicorp/nomad/issues/24292#issuecomment-2537078584
Ref: https://www.cni.dev/plugins/current/ipam/host-local/#files
* func: make paths relative
* func: make paths relative to the module inside the e2e terraform folder
* fix: add license files to gitignore
* func: move /etc and update all paths
* Uncomment forgotten code
* fix: update the path to the tls certificates to be local to the instance
The Nomad client can now optionally emit telemetry data from the
prerun and prestart hooks. This allows operators to monitor and
alert on failures and time taken to complete.
The new datapoints are:
- nomad.client.alloc_hook.prerun.success (counter)
- nomad.client.alloc_hook.prerun.failed (counter)
- nomad.client.alloc_hook.prerun.elapsed (sample)
- nomad.client.task_hook.prestart.success (counter)
- nomad.client.task_hook.prestart.failed (counter)
- nomad.client.task_hook.prestart.elapsed (sample)
The hook execution time is useful to Nomad engineering and will
help optimize code where possible and understand job specification
impacts on hook performance.
Currently only the PreRun and PreStart hooks have telemetry
enabled, so we limit the number of new metrics being produced.
Test setup for the `TestPeriodicDispatch_Add_TriggersUpdate` test can panic if
the goroutine for the runner is running concurrently with adding the job the
second time.
Update the test as follows:
* Make a copy when mutating the job before adding it.
* Add a lock around checking if the dispatcher has a waiting eval.
* Update to use `shoenig/test` in lieu of `testify`.
There are several places where we want to redact the secret ID of an ACL token,
some of which are in the Enterprise code base for Sentinel. Add a new method
`Sanitize` that mirrors the one we have on `Node`.
Ref: https://github.com/hashicorp/nomad-enterprise/pull/2087
* retain artifacts from test runs including test timing
* Pinning commit hashes for action helpers
* trigger for ui-test run
* Trying to isolate down to a simple upload
* Once more with mkdir
* What if we just wrote our own test reporter tho
* Let the partitioned runs handle placement
* Filter out common token logs, add a summary at the end, and note failures in logtime
* Custom reporter cannot also have an output file, he finds out two days late
* Aggregate summary, duration, and removing failure case
* Conditional test report generation
* Timeouts are errors
* Trying with un-partitioned input json file
* Remove the commented-out lines for main-only runs
* combine-ui-test-results as its own script
* Edit from Version functionality
* Reworked as Clone and Revert
* Change name warning for cloning as new job, version 0 checking, and erroring on sourceless clone
* If you try to plan a new version of a job with a different name of the one you're editing, suggest they might want to run a new one instead
* A few code comments and log cleanup
* Scaffolding new acceptance tests
* A whack of fun new tests
* Unit test for version number url passing on fetchRawDef
* Bit of cleanup
* fetchRawDefinition gets version support at adapter layer
* Handle spec-not-available-but-definition-is for clone-as-new-job
The retry_join logic was not allowing for retries to happen and
was exiting after the first failed discovery attempt. This change
fixes that behaviour and adds a test to ensure no further
regressions.
This PR adds Consul Template's executeTemplate function to the denylist by
default, in order to prevent accidental or malicious infinitely recursive
execution.
---------
Co-authored-by: Tim Gross <tgross@hashicorp.com>
A more comprehensive env.denylist that now includes more token, token file and
license variables.
---------
Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
Some plugins emit multiple topology segment entries for the same segment (ex. newer versions of AWS EBS) to accommodate convention changes in k8s. Check that segments are a superset instead of exactly equal to the plugin's topology segments.
