In #18754 we accidentally fixed a bug that prevented poststop tasks from getting
access to Variables. This was fixed in the 1.6.x branch in #19270, at which
point we discovered the fix had been done in main already as part of the auth
refactor. Add a changelog entry for it.
Clients prior to Nomad 1.7 cannot support the new workload identity-based
authentication to Consul and Vault. Add an implicit Nomad version constraint on
job submission for task groups that use the new workflow.
Includes a constraint test showing same-version prelease handling.
Some sections of the `consul` configuration are relevant only for clients or
servers. We updated our Vault docs to split these parameters out into their own
sections for clarity. Match that for the Consul docs.
The new Workload Identity workflow for Vault tokens correctly handles post-stop
tasks, however the legacy workflow does not. Attempts to get a Vault token are
rejected if the allocation is server-terminal or client-terminal, but we should
be waiting until the allocation is client-terminal (only) so that poststop tasks
get a chance to get Vault tokens too.
Fixes: https://github.com/hashicorp/nomad/issues/16886
Update the `nomad setup consul` command to include a `Selector` for the
`NamespaceRule` so the logic is only applied when the token has a claim
for `consul_namespace`.
Jobs without an explicit `consul.namespace` value receive a JWT without
the `consul_namespace` claim because Nomad is unable to determine which
Consul namespace should be used.
By using `NamespaceRules`, cluster operators are able to set a default
value for these jobs.
When porting the `ConsulTemplate` test, I made a last-minute refactor to the
assertions for waiting on files, and accidentally inverted the test assertion in
the process.
Also, when running `jobs3.Submit` you need to include the `Namespace` option so
that the cleanup function that gets return deletes the job from the correct
namespace. This was causing the namespace cleanup to fail because the job
deletion had failed.
* API command and jobspec docs
* PR comments addressed
* API docs for job/jobid/action socket
* Removing a perhaps incorrect origin of job_id across the jobs api doc
* PR comments addressed
In order to correctly handle Consul namespaces, auth methods and binding rules
must always be created in the default namespace only.
---------
Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>
and error more verbosely if it fails
also, add extra information to a failed evaluation
for more error visibility in other tests
---------
Co-authored-by: Juanadelacuesta <juanita.delacuestamorales@hashicorp.com>
In the legacy Consul token workflow, we check the user's token's permissions in
Consul at the time of job submit. The new task-level `consul` block was not
being respected when checking the list of namespaces.
The `script_check_hook` runs at the task level but can create script checks for
both task-level services and group-level services. Now that we allow the Consul
namespace to be set at the task-level `consul.namespace`, we need to have both
possible namespaces handy when creating and updating checks.
Refactor the JWT token derivation logic to only take a single request
since it was only ever called with a map of length one.
The original implementation received multiple requets to match the
legacy flow, but but legacy flow requests were batched from the Nomad
client to the server, which doesn't happen for JWT. Each JWT request
goes directly from the Nomad client to the Consul agent, so there is no
batching involved.
Token claims are used in several dynamic configuration in Consul and
Vault, such as Consul ACL bind and namespace rules, and Vault templated
policies.
Adding a claim for the Consul and Vault namespace defined for the
service or task allows cluster operators to create more flexible and
precise rules.
The `consul_namespace` claim is added to workload identities for Consul
services and to task workload identities that have the `consul_` name
prefix and are affected by a task or group `consul` block.
The `vault_namespace` claim is added to task workload identities that
have the `vault_` name prefix and are affected by a `vault` block.
When configuring Consul for multi-namespace support, the JWT auth method
needs to specify namespace rules. This attribute is set to `nil` in CE
but is used in Nomad ENT.
The task-level service hook is using the group-level method to get the provider
namespace, but this was not designed with task-level `consul` blocks in
mind. This leads to task-level services using the group-level
`consul.namespace`. Fix by creating a method to get the correct namespace and
move this into the service hook itself rather than in the outer `initHooks`
method.
The `-reschedule` flag stops allocations and assumes the Nomad scheduler
will create new allocations to replace them. But this is only true for
service and batch jobs.
Restarting non-service jobs with the `-reschedule` flag causes the
command to loop forever waiting for the allocations to be replaced,
which never happens.
Allocations for system jobs may be replaced by triggering an evaluation
after each stop to cause the reconciler to run again.
Sysbatch jobs should not be allowed to be rescheduled as they are never
replaced by the scheduler.
The `TestTemplateUpdateTriggers` is flaky because of what turned out to be
incompatibility between the Consul agent on the E2E cluster and the HCP Consul
server we were running but hadn't upgraded in a while. Upgrading the HCP Consul
server seems to have fixed the tests, but while I'm in here I've updated this
test suite:
* Port all the consul template test suite off of the old framework, and upgrade to
using e2e "v3" where feasible.
* Clean up some of the assertions in the update triggers test to make the
purpose of the test more clear.
* Remove unnecessary default fields from the job specs.
Closes: #19075
In #19194 the license was incorrectly changed to say that Nomad 1.6.2 or later
was under the BUSL, when in fact Nomad 1.6.2 and 1.6.3 are MPL2 licensed. Fix
this so that the correct (next!) version is shown as covered only.
fixes VaultSecrets test - it was failing due to a
regex mismatch (`^job` stopped matching when
copywrite headers got prepended to the jobspec).
but RegisterFromJobspec (which had the bug)
was only used in the one spot, so instead this
refactors the whole test to the v3 format
with testing.T and some additional fun stuff
that we can take advantage of with it.
some improvements:
* use a namespace
* use and extend existing test helpers
* add more test helpers
This simplifies the default setup of Nomad workloads WI-based
authentication for Consul by using a single auth method with 2 binding rules.
Users can still specify separate auth methods for services and tasks.
Nomad load all plugins from `plugin_dir` regardless if it is listed in
the agent configuration file. This can cause unexpected binaries to be
executed.
This commit begins the deprecation process of this behaviour. The Nomad
agent will emit a warning log for every plugin binary found without a
corresponding agent configuration block.
---------
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
Clarify the difference between the `/client` and `/node` endpoints and
link from one to the other to help users discover the endpoint they are
looking for.
Also update the `/client` page description and dynamic nod metadata
section headers to help the page be more discoverable by search engines.
