Commit Graph

26358 Commits

Author SHA1 Message Date
Kir Kolyshkin
d09c8ddf21 deps: switch to moby/sys/capability (#24093)
github.com/moby/sys/capability is a fork of the (no longer maintained)
github.com/syndtr/gocapability package.

For changes since the fork took place, see
https://github.com/moby/sys/blob/main/capability/CHANGELOG.md

Note that the "workaround for RHEL6" is removed for a number of reasons.
Feel free to choose the one you like the most, either is sufficient:

1. /proc/sys/kernel/cap_last_cap is available since RHEL 6.7
   (kernel 2.6.32-573.el6), released 9 years ago (2015-07-22).

2. It incorrectly returns CAP_BLOCK_SUSPEND (36), which was only added
   in kernel v3.5 and was never backported to RHEL6 kernels. The
   correct value for RHEL6 would be CAP_MAC_ADMIN (33).

3. As far as upstream kernels go, /proc/sys/kernel/cap_last_cap was
   added in kernel v3.2, and a correct value depends on the kernel
   version. It could be CAP_WAKE_ALARM (35), added to kernel v3.0, or
   CAP_SYSLOG (34), added to kernel v2.6.38, or possibly a lesser value
   for even older kernels.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2024-11-11 14:07:31 -05:00
Piotr Kazmierczak
1f9e97b198 Merge pull request #24424 from hashicorp/post-1.9.3-release
Post 1.9.3 release
2024-11-11 19:45:39 +01:00
hc-github-team-nomad-core
ff699c5e06 Prepare for next release 2024-11-11 19:40:45 +01:00
hc-github-team-nomad-core
9f9e66fa61 Generate files for 1.9.3 release 2024-11-11 19:40:44 +01:00
Piotr Kazmierczak
9e8b3e5c04 changelog for 24412 (#24421) 2024-11-11 17:28:06 +01:00
Tim Gross
645923274d fsm: fix bug in snapshot restore for removed timetable (#24412)
When we removed the time table in #24112 we introduced a bug where if a previous
version of Nomad had written a time table entry, we'd return from the restore
loop early and never load the rest of the FSM. This will result in a mostly or
partially wiped state for that Nomad node, which would then be out of sync with
its peers (which would also have the same problem on upgrade).

The bug only occurs when the FSM is being restored from snapshot, which isn't
the case if you test with a server that's only written Raft logs and not
snapshotted them.

While fixing this bug, we still need to ensure we're reading the time table
entries even if we're throwing them away, so that we move the snapshot reader
along to the next full entry.

Fixes: https://github.com/hashicorp/nomad/issues/24411
2024-11-11 08:01:18 -05:00
Michael Smithhisler
0714353324 fix: handle template re-renders on client restart (#24399)
When multiple templates with api functions are included in a task, it's
possible for consul-template to re-render templates as it creates
watchers, overwriting render event data. This change uses event fields
that do not get overwritten, and only executes the change mode for
templates that were actually written to disk.

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-11-08 12:49:38 -05:00
Piotr Kazmierczak
ccba08a9e6 release: fix typo in the LAST_RELEASE sed regex (#24403) 2024-11-08 17:39:06 +01:00
Juana De La Cuesta
dfa0066d06 [gh-24311] Expand on documentation about jobs that are both parameterised and periodic (#24384)
* docs: expand on documentation about jobs that are both parameterized and periodic

* fix: typo

* docs: expand on the example

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/periodic.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/periodic.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* style: improve the content with PR suggestions

* periodic.mdx fix link to parameterized

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update parameterized.mdx

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update parameterized.mdx

---------

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>
2024-11-08 17:29:46 +01:00
Piotr Kazmierczak
97ea684a43 Merge pull request #24402 from hashicorp/post-1.9.2-release
Post 1.9.2 release
2024-11-08 15:26:52 +01:00
Piotr Kazmierczak
8b5c88bb63 Merge release 1.9.2 files 2024-11-08 15:24:12 +01:00
Piotr Kazmierczak
d41082a2cc changelog correction 2024-11-08 15:21:39 +01:00
hc-github-team-nomad-core
9e29212920 Prepare for next release 2024-11-08 15:21:39 +01:00
hc-github-team-nomad-core
1938a7578b Generate files for 1.9.2 release 2024-11-08 15:21:39 +01:00
dependabot[bot]
732f608c06 chore(deps): bump golang.org/x/sys from 0.25.0 to 0.26.0 (#24138) 2024-11-08 14:06:58 +00:00
James Rasell
ff7815e4fd admin: add GitHub PR template. (#24365)
The template aims to ensure all PRs include the required
information for reviewers. The checklist items help ensure merging
happens quickly and in the correct manner.

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-11-08 10:09:33 +00:00
Piotr Kazmierczak
c5249c6ca4 gc: be consistent with setting create/modify timestamp tz (#24389)
Whenever setting objects creation/modify time, we should always use UTC. #24112
introduced some inconsistencies in this area, and this PR fixes it.
2024-11-07 22:53:54 +01:00
Tim Gross
30849c518e CSI: fix namespace ACL bypass on create/register APIs (#24396)
When creating or registering a CSI volume, the RPC handler uses the volume
specification's namespace instead of the request namespace. This works as
intended, but the ACL check is only on the request namespace.

This allows a cross-namespace ACL bypass for authenticated users who have
`csi-write-volume` capabilities in one namespace but not another namespace. Such
a user can set the volume specification to a forbidden namespace while setting
the `-namespace` flag in the CLI or API. The ACL check happens against the
namespace they do have permission to, but the volume is created in the forbidden
namespace.

This changeset fixes the bug by moving the namespace check into the loop over
the volumes being written by the RPCs. It also updates the tests to better cover
ACL checking in these two RPCs.

Ref: CVE-2024-10975
Ref: https://hashicorp.atlassian.net/browse/SECVULN-15463
Fixes: https://github.com/hashicorp/nomad/issues/24397
2024-11-07 14:47:30 -05:00
Phil Renaud
3d9003879e [ui] Update the Task Lifecycle Status chart (#24133)
* Updates the Task Lifecycle Status chart to show which pre/poststart task may have failed

* Default colour to prevent HDS error

* De-duplicated data-test attr and added is-active and is-finished test classes

* Failed and Pending state tests
2024-11-07 13:57:58 -05:00
Seth Hoenig
a0ff07393b drivers: provide empty implementations of cgroup helpers for non-root nomad (#24392) 2024-11-07 12:24:37 -06:00
Daniel Bennett
a036b75aef api: new dispatch endpoint sends body as Payload (#24381)
this opens up dispatching parameterized jobs by systems
that do not allow modifying what http request body they send

e.g. these two things are equal:

POST '{"Payload": "'"$(base64 <<< "hello")"'"}' /v1/job/my-job/dispatch
POST 'hello' /v1/job/my-job/dispatch/payload
2024-11-07 10:12:29 -06:00
James Rasell
316430b094 keyring: Fix a panic when decrypting aead with empty RSA block. (#24383)
Clusters that have gone through several upgrades have be found to
include keyring material which has an empty RSA block.

In more recent versions of Nomad, an empty RSA block is omitted
from being written to disk. This results in the panic not being
present. Older versions, however, did not have this struct tag
meaning we wrote an empty JSON block which is not accounted for
in the current version.
2024-11-07 16:08:18 +00:00
Phil Renaud
498b29b3cf [ui] When your token expires and you sign in again, redirect to your original route. (#24374)
* Upon sign-in post-expiry/403, redirect to original route

* Tests for token expiry re-routing

* Had made one of the new test tokens a management token, which conflicted with another test but was not necessary
2024-11-07 10:43:43 -05:00
Seth Hoenig
4ef4bebd1f connect: handle grpc_address as gosockaddr/template string (#24280)
* connect: handle grpc_address as gosockaddr/template string

This PR fixes a bug where the consul.grpc_address could not be set using
a go-sockaddr/template string. This was inconsistent with how we do accept
such strings for consul.address values.

* add changelog
2024-11-07 09:04:58 -06:00
Seth Hoenig
b58abf48c1 drivers: move executor process out of v1 task cgroup after process starts (#24340)
* drivers: move executor process out of v1 task cgroup after process starts

This PR changes the behavior of the raw exec task driver on old cgroups v1
systems such that the executor process is no longer a member of the cgroups
created for the task. Now, the executor process is placed into those
cgroups and starts the task child process (just as before), but now then
exits those cgroups and exists in the nomad parent cgroup. This change
makes the behavior sort of similar to cgroups v2 systems, where we never
have the executor enter the task cgroup to begin with (because we can
directly clone(3) the task process into it).

Fixes #23951

* executor: handle non-linux case

* cgroups: add test case for no executor process in task cgroup (v1)

* add changelog

* drivers: also move executor out of cpuset cgroup
2024-11-07 07:31:38 -06:00
Piotr Kazmierczak
73383ee755 e2e: unflake testDockerExecStdin (#24385) 2024-11-07 13:35:32 +01:00
James Rasell
c44f933aeb test: ensure RPC only test client sets enterprise specific config. (#24376) 2024-11-06 13:43:25 +00:00
James Rasell
b4c397e8e4 ci: ensure golangci-lint is run with Go build tags. (#24364) 2024-11-06 11:12:40 +00:00
dependabot[bot]
bdee6f04a5 chore(deps): bump github.com/moby/sys/mountinfo from 0.7.1 to 0.7.2 (#24136) 2024-11-06 07:49:24 +00:00
Tim Gross
a8b84a6eed testing: RPC-only test client helper (#24371)
In #10193 we introduced a testing helper that spins up a client RPC server
without the rest of the client operations so that we can make server-side client
RPC tests lighter. But this wasn't actually ever wired up to the intended
target. While working on Dynamic Host Volumes I noticed that this would be
useful for RPC tests.

This changeset fixes some bugs in the helper that arose from client code drift,
and makes it used by the client RPC tests for CSI. This will also get used for
the DHV RPC tests.

Ref: https://github.com/hashicorp/nomad/pull/10193
2024-11-05 14:59:53 -05:00
Daniel Bennett
c32d9ed6f5 docs: ipv6: small fixes (#24368)
* escaping newlines is not allowed in go-sockaddr template
* client{} block in client section
* tiny extra clarification that the NOMAD_ADDR is an example
2024-11-05 11:11:36 -06:00
James Rasell
155eac50e9 ci: Fix retrieving of Vault secrets for build workflow. (#24366) 2024-11-05 16:00:19 +00:00
dependabot[bot]
60439142c2 chore(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (#24360) 2024-11-05 09:27:52 +00:00
Martijn Vegter
8545e1c79f scheduler: fixed a bug where resource calculation did not account correctly for poststart tasks (#24297)
Fixes a bug in the AllocatedResources.Comparable method, which resulted in
reporting less required resources than actually expected. This could result in
overscheduling of allocations on a single node  and overlapping cgroup cpusets.
2024-11-05 09:07:15 +00:00
dependabot[bot]
f75e2c276e chore(deps): bump actions/checkout from 4.2.1 to 4.2.2 (#24298) 2024-11-04 11:42:37 +00:00
Seth Hoenig
4dbcdd103a ui: show region in header gutter when only one region exists (#24320)
* ui: show region in header gutter when only one region exists

This PR adds a plain text label of the region to the header when there is
only one region present. Before, nothing was showin in this case, and a
dropdown was shown on federated clusters.

The use case here is for operators of multiple non-federated Nomad clusters,
when all the UI's involved otherwise look identical.

* [ui] Signing in with a token explicitly sets the region dropdown activeRegion (#24347)

* Signing in with a token explicitly sets the region dropdown activeREgion

* Test and Select a Region default text

* Account for 403 on mocked agent members req

* Dont show the region if it isnt set in agent config

* Small padding css change

* unit test condition moved to stubbable acceptance test

---------

Co-authored-by: Phil Renaud <phil.renaud@hashicorp.com>
2024-11-01 22:58:13 -04:00
Piotr Kazmierczak
f7847c6e5b state: remove TimeTable and rely on objects' modify times instead (#24112)
Core scheduler relies on a special table in the state store—the TimeTable—to
figure out which objects can be GC'd. The TimeTable correlates Raft indices
with objects insertion time, a solution we used before most of the objects we
store in the state contained timestamps. This introduced a bit of a memory
overhead and complexity, but most importantly meant that any GC threshold users
set greater than timeTableLimit = 72 * time.Hour was ignored. This PR removes
the TimeTable and relies on object timestamps to determine whether they could
be GCd or not.
2024-11-01 19:38:04 +01:00
Michael Smithhisler
0f97574eae test: fix rawexec driver unix test imports (#24352) 2024-11-01 12:10:03 -04:00
Michael Smithhisler
658c429d75 Drivers: add work_dir config to exec/raw_exec/java drivers (#24249)
---------

Co-authored-by: wurosh <uros.m.perisic@gmail.com>
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-11-01 11:04:40 -04:00
James Rasell
58ea294f0b docs: add note to reschedule block for update progress deadline. (#24346)
Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>
2024-11-01 14:54:51 +00:00
Juana De La Cuesta
c18418fa61 Merge pull request #20073 from hashicorp/feat/uid-gid-restriction
Adds ability to restrict uid and gids in exec and raw_exec
2024-10-31 15:48:45 +01:00
Juanadelacuesta
80e398bbf7 test: add tests for validateBounds 2024-10-31 14:54:27 +01:00
Juanadelacuesta
d0b015ec01 func: move the user andd group type declarations 2024-10-31 10:34:26 +01:00
Juanadelacuesta
8752bb0a65 func: move the user lookup into the validation, it's used everywhere the function is called 2024-10-31 10:34:26 +01:00
Juana De La Cuesta
3449056cd6 Update website/content/docs/drivers/raw_exec.mdx
Co-authored-by: Michael Smithhisler <michael.smithhisler@hashicorp.com>
2024-10-31 10:26:26 +01:00
Juana De La Cuesta
3f32557f1e Update website/content/docs/drivers/exec.mdx
Co-authored-by: Michael Smithhisler <michael.smithhisler@hashicorp.com>
2024-10-31 09:43:49 +01:00
Juana De La Cuesta
f1439f54f7 Update drivers/shared/validators/validators.go
Co-authored-by: James Rasell <jrasell@users.noreply.github.com>
2024-10-31 09:32:51 +01:00
Juanadelacuesta
3f884bb3fa fix: remove the setConfig and modify the test driver to include idValidator to avoid panics 2024-10-30 17:38:54 +01:00
Juanadelacuesta
f954a1a5e8 fix: remove the setConfig and modify the test driver to include idValidator to avoid panics 2024-10-30 16:16:42 +01:00
Juanadelacuesta
a86e951f03 style: rename DeniedHostGidsStr to reflect refactor 2024-10-30 15:22:50 +01:00