Commit Graph

26371 Commits

Author SHA1 Message Date
Tim Gross
ff8ca8a4c5 tools: filter Nomad Enterprise tags in pre-push hook (#24452)
Our git pre-push hook already prevents Nomad Enterprise code from getting pushed
anywhere but its own repo. But this hook only works for files on the current
worktree (checkout). Were you to fetch an Enterprise tag into your local
Community Edition repo but not have it checked out, and then `git push --tags`,
you'd push that tag and the associated commit history.

Add tag filtering to the pre-push hook to prevent Enterprise tags (and the older
`+pro` SKU) tags from getting pushed to the Community Edition repo.
2024-11-13 09:50:43 -05:00
Bob Conan
cf45f1836f Update CHANGELOG.md, fix typo(s) (#24449)
* Updated CHANGELOG.md, fix typo(s)

* Update CHANGELOG.md, fix typos
2024-11-12 20:26:20 -05:00
Michael Schurter
8dd570d6ca docs: upgrade docs should point at real version (#24438)
Let users know what happened to 1.9.2 but label the gc change as the
first working release (1.9.3).
2024-11-12 11:05:27 -08:00
Seth Hoenig
b36c77781f build: update to go1.23.3 (#24419)
https://github.com/golang/go/issues?q=milestone%3AGo1.23.3+label%3ACherryPickApproved
2024-11-12 10:59:02 -06:00
James Rasell
4e7496d246 keyring: Fix a panic when decrypting aead with empty RSA block. (#24442)
Clusters that have gone through several upgrades have be found
to include keyring material which has an empty RSA block.

In more recent versions of Nomad, an empty RSA block is omitted
from being written to disk. This results in the panic not being
present. Older versions, however, did not have this struct tag
meaning we wrote an empty JSON block which is not accounted for
in the current version.
2024-11-12 14:26:35 +00:00
Piotr Kazmierczak
7a9e3d2487 fsm: handle EventSinkSnapshot type correctly (#24418)
We never handled EventSinkSnapshot correctly. We should error on this snapshot
type, since it should never be encountered in prod Nomad clusters.
2024-11-12 10:42:19 +01:00
Eduardo Medeiros
f8c85b036b docs: remove duplicated word. (#24433)
remove duplicated word “Using using”
2024-11-11 16:10:10 -05:00
Tim Gross
189d648f95 csi: remove redundant namespace field from volume status output (#24432)
The `volume status :id` command outputs the namespace for a CSI volume
twice. Drop the second output.

Ref: https://github.com/hashicorp/nomad/pull/24382#discussion_r1837097250
2024-11-11 16:05:59 -05:00
Shantanu Gadgil
1d123fdc7b docs/tools: add Nomad Tools (#24416)
Add Nomad Tools

`https://github.com/Kamilcuk/nomad-tools`
2024-11-11 14:22:46 -05:00
dependabot[bot]
af379f7020 chore(deps): bump github.com/docker/cli (#24414)
Bumps [github.com/docker/cli](https://github.com/docker/cli) from 24.0.6+incompatible to 27.3.1+incompatible.
- [Commits](https://github.com/docker/cli/compare/v24.0.6...v27.3.1)

---
updated-dependencies:
- dependency-name: github.com/docker/cli
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 14:21:09 -05:00
dependabot[bot]
cfe17e6e8b chore(deps): bump github.com/prometheus/common from 0.45.0 to 0.60.1 (#24303)
Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.45.0 to 0.60.1.
- [Release notes](https://github.com/prometheus/common/releases)
- [Changelog](https://github.com/prometheus/common/blob/main/RELEASE.md)
- [Commits](https://github.com/prometheus/common/compare/v0.45.0...v0.60.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 14:17:25 -05:00
dependabot[bot]
c688322a3f chore(deps): bump github.com/aws/aws-sdk-go from 1.44.210 to 1.55.5 (#24135)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.210 to 1.55.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.210...v1.55.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 14:15:23 -05:00
dependabot[bot]
a8676c6e67 chore(deps): bump github.com/hashicorp/consul/api from 1.29.1 to 1.30.0 (#24262)
Bumps [github.com/hashicorp/consul/api](https://github.com/hashicorp/consul) from 1.29.1 to 1.30.0.
- [Release notes](https://github.com/hashicorp/consul/releases)
- [Changelog](https://github.com/hashicorp/consul/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/consul/compare/api/v1.29.1...api/v1.30.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/consul/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 14:10:16 -05:00
Kir Kolyshkin
d09c8ddf21 deps: switch to moby/sys/capability (#24093)
github.com/moby/sys/capability is a fork of the (no longer maintained)
github.com/syndtr/gocapability package.

For changes since the fork took place, see
https://github.com/moby/sys/blob/main/capability/CHANGELOG.md

Note that the "workaround for RHEL6" is removed for a number of reasons.
Feel free to choose the one you like the most, either is sufficient:

1. /proc/sys/kernel/cap_last_cap is available since RHEL 6.7
   (kernel 2.6.32-573.el6), released 9 years ago (2015-07-22).

2. It incorrectly returns CAP_BLOCK_SUSPEND (36), which was only added
   in kernel v3.5 and was never backported to RHEL6 kernels. The
   correct value for RHEL6 would be CAP_MAC_ADMIN (33).

3. As far as upstream kernels go, /proc/sys/kernel/cap_last_cap was
   added in kernel v3.2, and a correct value depends on the kernel
   version. It could be CAP_WAKE_ALARM (35), added to kernel v3.0, or
   CAP_SYSLOG (34), added to kernel v2.6.38, or possibly a lesser value
   for even older kernels.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2024-11-11 14:07:31 -05:00
Piotr Kazmierczak
1f9e97b198 Merge pull request #24424 from hashicorp/post-1.9.3-release
Post 1.9.3 release
2024-11-11 19:45:39 +01:00
hc-github-team-nomad-core
ff699c5e06 Prepare for next release 2024-11-11 19:40:45 +01:00
hc-github-team-nomad-core
9f9e66fa61 Generate files for 1.9.3 release 2024-11-11 19:40:44 +01:00
Piotr Kazmierczak
9e8b3e5c04 changelog for 24412 (#24421) 2024-11-11 17:28:06 +01:00
Tim Gross
645923274d fsm: fix bug in snapshot restore for removed timetable (#24412)
When we removed the time table in #24112 we introduced a bug where if a previous
version of Nomad had written a time table entry, we'd return from the restore
loop early and never load the rest of the FSM. This will result in a mostly or
partially wiped state for that Nomad node, which would then be out of sync with
its peers (which would also have the same problem on upgrade).

The bug only occurs when the FSM is being restored from snapshot, which isn't
the case if you test with a server that's only written Raft logs and not
snapshotted them.

While fixing this bug, we still need to ensure we're reading the time table
entries even if we're throwing them away, so that we move the snapshot reader
along to the next full entry.

Fixes: https://github.com/hashicorp/nomad/issues/24411
2024-11-11 08:01:18 -05:00
Michael Smithhisler
0714353324 fix: handle template re-renders on client restart (#24399)
When multiple templates with api functions are included in a task, it's
possible for consul-template to re-render templates as it creates
watchers, overwriting render event data. This change uses event fields
that do not get overwritten, and only executes the change mode for
templates that were actually written to disk.

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-11-08 12:49:38 -05:00
Piotr Kazmierczak
ccba08a9e6 release: fix typo in the LAST_RELEASE sed regex (#24403) 2024-11-08 17:39:06 +01:00
Juana De La Cuesta
dfa0066d06 [gh-24311] Expand on documentation about jobs that are both parameterised and periodic (#24384)
* docs: expand on documentation about jobs that are both parameterized and periodic

* fix: typo

* docs: expand on the example

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/periodic.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/periodic.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* style: improve the content with PR suggestions

* periodic.mdx fix link to parameterized

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update parameterized.mdx

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update website/content/docs/job-specification/parameterized.mdx

Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>

* Update parameterized.mdx

---------

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
Co-authored-by: Aimee Ukasick <aimee.ukasick@hashicorp.com>
2024-11-08 17:29:46 +01:00
Piotr Kazmierczak
97ea684a43 Merge pull request #24402 from hashicorp/post-1.9.2-release
Post 1.9.2 release
2024-11-08 15:26:52 +01:00
Piotr Kazmierczak
8b5c88bb63 Merge release 1.9.2 files 2024-11-08 15:24:12 +01:00
Piotr Kazmierczak
d41082a2cc changelog correction 2024-11-08 15:21:39 +01:00
hc-github-team-nomad-core
9e29212920 Prepare for next release 2024-11-08 15:21:39 +01:00
hc-github-team-nomad-core
1938a7578b Generate files for 1.9.2 release 2024-11-08 15:21:39 +01:00
dependabot[bot]
732f608c06 chore(deps): bump golang.org/x/sys from 0.25.0 to 0.26.0 (#24138) 2024-11-08 14:06:58 +00:00
James Rasell
ff7815e4fd admin: add GitHub PR template. (#24365)
The template aims to ensure all PRs include the required
information for reviewers. The checklist items help ensure merging
happens quickly and in the correct manner.

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-11-08 10:09:33 +00:00
Piotr Kazmierczak
c5249c6ca4 gc: be consistent with setting create/modify timestamp tz (#24389)
Whenever setting objects creation/modify time, we should always use UTC. #24112
introduced some inconsistencies in this area, and this PR fixes it.
2024-11-07 22:53:54 +01:00
Tim Gross
30849c518e CSI: fix namespace ACL bypass on create/register APIs (#24396)
When creating or registering a CSI volume, the RPC handler uses the volume
specification's namespace instead of the request namespace. This works as
intended, but the ACL check is only on the request namespace.

This allows a cross-namespace ACL bypass for authenticated users who have
`csi-write-volume` capabilities in one namespace but not another namespace. Such
a user can set the volume specification to a forbidden namespace while setting
the `-namespace` flag in the CLI or API. The ACL check happens against the
namespace they do have permission to, but the volume is created in the forbidden
namespace.

This changeset fixes the bug by moving the namespace check into the loop over
the volumes being written by the RPCs. It also updates the tests to better cover
ACL checking in these two RPCs.

Ref: CVE-2024-10975
Ref: https://hashicorp.atlassian.net/browse/SECVULN-15463
Fixes: https://github.com/hashicorp/nomad/issues/24397
2024-11-07 14:47:30 -05:00
Phil Renaud
3d9003879e [ui] Update the Task Lifecycle Status chart (#24133)
* Updates the Task Lifecycle Status chart to show which pre/poststart task may have failed

* Default colour to prevent HDS error

* De-duplicated data-test attr and added is-active and is-finished test classes

* Failed and Pending state tests
2024-11-07 13:57:58 -05:00
Seth Hoenig
a0ff07393b drivers: provide empty implementations of cgroup helpers for non-root nomad (#24392) 2024-11-07 12:24:37 -06:00
Daniel Bennett
a036b75aef api: new dispatch endpoint sends body as Payload (#24381)
this opens up dispatching parameterized jobs by systems
that do not allow modifying what http request body they send

e.g. these two things are equal:

POST '{"Payload": "'"$(base64 <<< "hello")"'"}' /v1/job/my-job/dispatch
POST 'hello' /v1/job/my-job/dispatch/payload
2024-11-07 10:12:29 -06:00
James Rasell
316430b094 keyring: Fix a panic when decrypting aead with empty RSA block. (#24383)
Clusters that have gone through several upgrades have be found to
include keyring material which has an empty RSA block.

In more recent versions of Nomad, an empty RSA block is omitted
from being written to disk. This results in the panic not being
present. Older versions, however, did not have this struct tag
meaning we wrote an empty JSON block which is not accounted for
in the current version.
2024-11-07 16:08:18 +00:00
Phil Renaud
498b29b3cf [ui] When your token expires and you sign in again, redirect to your original route. (#24374)
* Upon sign-in post-expiry/403, redirect to original route

* Tests for token expiry re-routing

* Had made one of the new test tokens a management token, which conflicted with another test but was not necessary
2024-11-07 10:43:43 -05:00
Seth Hoenig
4ef4bebd1f connect: handle grpc_address as gosockaddr/template string (#24280)
* connect: handle grpc_address as gosockaddr/template string

This PR fixes a bug where the consul.grpc_address could not be set using
a go-sockaddr/template string. This was inconsistent with how we do accept
such strings for consul.address values.

* add changelog
2024-11-07 09:04:58 -06:00
Seth Hoenig
b58abf48c1 drivers: move executor process out of v1 task cgroup after process starts (#24340)
* drivers: move executor process out of v1 task cgroup after process starts

This PR changes the behavior of the raw exec task driver on old cgroups v1
systems such that the executor process is no longer a member of the cgroups
created for the task. Now, the executor process is placed into those
cgroups and starts the task child process (just as before), but now then
exits those cgroups and exists in the nomad parent cgroup. This change
makes the behavior sort of similar to cgroups v2 systems, where we never
have the executor enter the task cgroup to begin with (because we can
directly clone(3) the task process into it).

Fixes #23951

* executor: handle non-linux case

* cgroups: add test case for no executor process in task cgroup (v1)

* add changelog

* drivers: also move executor out of cpuset cgroup
2024-11-07 07:31:38 -06:00
Piotr Kazmierczak
73383ee755 e2e: unflake testDockerExecStdin (#24385) 2024-11-07 13:35:32 +01:00
James Rasell
c44f933aeb test: ensure RPC only test client sets enterprise specific config. (#24376) 2024-11-06 13:43:25 +00:00
James Rasell
b4c397e8e4 ci: ensure golangci-lint is run with Go build tags. (#24364) 2024-11-06 11:12:40 +00:00
dependabot[bot]
bdee6f04a5 chore(deps): bump github.com/moby/sys/mountinfo from 0.7.1 to 0.7.2 (#24136) 2024-11-06 07:49:24 +00:00
Tim Gross
a8b84a6eed testing: RPC-only test client helper (#24371)
In #10193 we introduced a testing helper that spins up a client RPC server
without the rest of the client operations so that we can make server-side client
RPC tests lighter. But this wasn't actually ever wired up to the intended
target. While working on Dynamic Host Volumes I noticed that this would be
useful for RPC tests.

This changeset fixes some bugs in the helper that arose from client code drift,
and makes it used by the client RPC tests for CSI. This will also get used for
the DHV RPC tests.

Ref: https://github.com/hashicorp/nomad/pull/10193
2024-11-05 14:59:53 -05:00
Daniel Bennett
c32d9ed6f5 docs: ipv6: small fixes (#24368)
* escaping newlines is not allowed in go-sockaddr template
* client{} block in client section
* tiny extra clarification that the NOMAD_ADDR is an example
2024-11-05 11:11:36 -06:00
James Rasell
155eac50e9 ci: Fix retrieving of Vault secrets for build workflow. (#24366) 2024-11-05 16:00:19 +00:00
dependabot[bot]
60439142c2 chore(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (#24360) 2024-11-05 09:27:52 +00:00
Martijn Vegter
8545e1c79f scheduler: fixed a bug where resource calculation did not account correctly for poststart tasks (#24297)
Fixes a bug in the AllocatedResources.Comparable method, which resulted in
reporting less required resources than actually expected. This could result in
overscheduling of allocations on a single node  and overlapping cgroup cpusets.
2024-11-05 09:07:15 +00:00
dependabot[bot]
f75e2c276e chore(deps): bump actions/checkout from 4.2.1 to 4.2.2 (#24298) 2024-11-04 11:42:37 +00:00
Seth Hoenig
4dbcdd103a ui: show region in header gutter when only one region exists (#24320)
* ui: show region in header gutter when only one region exists

This PR adds a plain text label of the region to the header when there is
only one region present. Before, nothing was showin in this case, and a
dropdown was shown on federated clusters.

The use case here is for operators of multiple non-federated Nomad clusters,
when all the UI's involved otherwise look identical.

* [ui] Signing in with a token explicitly sets the region dropdown activeRegion (#24347)

* Signing in with a token explicitly sets the region dropdown activeREgion

* Test and Select a Region default text

* Account for 403 on mocked agent members req

* Dont show the region if it isnt set in agent config

* Small padding css change

* unit test condition moved to stubbable acceptance test

---------

Co-authored-by: Phil Renaud <phil.renaud@hashicorp.com>
2024-11-01 22:58:13 -04:00
Piotr Kazmierczak
f7847c6e5b state: remove TimeTable and rely on objects' modify times instead (#24112)
Core scheduler relies on a special table in the state store—the TimeTable—to
figure out which objects can be GC'd. The TimeTable correlates Raft indices
with objects insertion time, a solution we used before most of the objects we
store in the state contained timestamps. This introduced a bit of a memory
overhead and complexity, but most importantly meant that any GC threshold users
set greater than timeTableLimit = 72 * time.Hour was ignored. This PR removes
the TimeTable and relies on object timestamps to determine whether they could
be GCd or not.
2024-11-01 19:38:04 +01:00