Commit Graph

26019 Commits

Author SHA1 Message Date
Tim Gross
c8be863bc8 reporting: allow export interval and address to be configurable (#23674)
The go-census library supports configuration to send metrics to a local
development version of the collector. Add "undocumented" configuration options
to the `reporting` block allow developers to debug and verify we're sending the
data we expect with real Nomad servers and not just unit tests.

Ref: https://hashicorp.atlassian.net/browse/NET-10057
Ref: https://github.com/hashicorp/nomad-enterprise/pull/1708
2024-07-24 08:29:59 -04:00
Tim Gross
c280891703 template: allow change_mode script to run after client restart (#23663)
For templates with `change_mode = "script"`, we set a driver handle in the
poststart method, so the template runner can execute the script inside the
task. But when the client is restarted and the template contents change during
that window, we trigger a change_mode in the prestart method. In that case, the
hook will not have the handle and so returns an errror trying to run the change
mode.

We restore the driver handle before we call any prestart hooks, so we can pass
that handle in the constructor whenever it's available. In the normal task start
case the handle will be empty but also won't be called.

The error messages are also misleading, as there's no capabilities check
happening here. Update the error messages to match.

Fixes: https://github.com/hashicorp/nomad/issues/15851
Ref: https://hashicorp.atlassian.net/browse/NET-9338
2024-07-24 08:29:39 -04:00
Deniz Onur Duzgun
7a2c70e3f6 deps: bump azidentity to v1.7.0 (#23664) 2024-07-22 15:03:19 -04:00
Daniel Bennett
10d3f1749b e2e: test all cni config formats (#23650) 2024-07-22 10:17:03 -05:00
Tim Gross
0f4014b4a9 docs: external KMS configuration (#23600)
In #23580 we're implementing support for encrypting Nomad's key material with
external KMS providers or Vault Transit. This changeset breaks out the
documentation from that PR to keep the review manageable and present it to a
wider set of reviewers.

Ref: https://hashicorp.atlassian.net/browse/NET-10334
Ref: https://github.com/hashicorp/nomad/issues/14852
Ref: https://github.com/hashicorp/nomad/pull/23580
2024-07-19 15:08:54 -04:00
Tim Gross
a29f9b6fc0 keyring: E2E testing for KMS/rotation (#23601)
In #23580 we're implementing support for encrypting Nomad's key material with
external KMS providers or Vault Transit. This changeset breaks out the E2E
infrastructure and testing from that PR to keep the review manageable.

Ref: https://hashicorp.atlassian.net/browse/NET-10334
Ref: https://github.com/hashicorp/nomad/issues/14852
Ref: https://github.com/hashicorp/nomad/pull/23580
2024-07-19 13:49:48 -04:00
Tim Gross
2f4353412d keyring: support prepublishing keys (#23577)
When a root key is rotated, the servers immediately start signing Workload
Identities with the new active key. But workloads may be using those WI tokens
to sign into external services, which may not have had time to fetch the new
public key and which might try to fetch new keys as needed.

Add support for prepublishing keys. Prepublished keys will be visible in the
JWKS endpoint but will not be used for signing or encryption until their
`PublishTime`. Update the periodic key rotation to prepublish keys at half the
`root_key_rotation_threshold` window, and promote prepublished keys to active
after the `PublishTime`.

This changeset also fixes two bugs in periodic root key rotation and garbage
collection, both of which can't be safely fixed without implementing
prepublishing:

* Periodic root key rotation would never happen because the default
  `root_key_rotation_threshold` of 720h exceeds the 72h maximum window of the FSM
  time table. We now compare the `CreateTime` against the wall clock time instead
  of the time table. (We expect to remove the time table in future work, ref
  https://github.com/hashicorp/nomad/issues/16359)
* Root key garbage collection could GC keys that were used to sign
  identities. We now wait until `root_key_rotation_threshold` +
  `root_key_gc_threshold` before GC'ing a key.
* When rekeying a root key, the core job did not mark the key as inactive after
  the rekey was complete.

Ref: https://hashicorp.atlassian.net/browse/NET-10398
Ref: https://hashicorp.atlassian.net/browse/NET-10280
Fixes: https://github.com/hashicorp/nomad/issues/19669
Fixes: https://github.com/hashicorp/nomad/issues/23528
Fixes: https://github.com/hashicorp/nomad/issues/19368
2024-07-19 13:29:41 -04:00
Piotr Kazmierczak
78bc8e7843 scheduler: fix TestAllocSet_filterByTainted (#23648) 2024-07-19 17:41:06 +02:00
Tim Gross
a8ab2d13b4 docs: explain how to use insecure registries with Docker (#23642)
The documentation for the `SSL` option for the Docker driver is
misleading inasmuch as it's both deprecated and non-functional in current
versions of Docker. Remove this option from the docs and add a section
explaining how to use insecure registries.

Fixes: https://github.com/hashicorp/nomad/issues/23616
2024-07-19 11:18:47 -04:00
Phil Renaud
5df6fe6a57 [ui] Pack metadata booleanified and added to the statuses endpoint (#23404)
* Pack metadata booleanified and added to the statuses endpoint

* Simplify job.IsPack declaration
2024-07-19 11:16:43 -04:00
dependabot[bot]
cf6ce224b3 chore(deps): bump github.com/hashicorp/go-checkpoint (#23588) 2024-07-19 15:13:43 +01:00
James Rasell
de0a86a55a docs: Fix ACL login API path documentation. (#23624) 2024-07-19 11:56:15 +01:00
Piotr Kazmierczak
f81b72e0bf exec2: update documentation with oom_score_adj settings (#23327) 2024-07-19 09:09:27 +02:00
Juana De La Cuesta
4f99d2b7f8 Merge pull request #23617 from hashicorp/nj-changelog
Add the missing v on the changelog call
2024-07-18 22:47:33 +02:00
Daniel Bennett
32d8ec446f cni: fix parsing .conf and .json configs (#23629)
so we can support more than just .conflist format
like our docs claim we do
2024-07-18 14:55:13 -05:00
Tim Gross
857f20e0ba deps: update go-kms-wrapping and Azure SDK (#23622)
I'm pulling this out to a shared PR between the two, because it'll make
backporting easier.

Closes: #23621
Closes: #23589
2024-07-18 10:18:42 -04:00
Tim Gross
c970d22164 keyring: support external KMS for key encryption key (KEK) (#23580)
In Nomad 1.4.0, we shipped support for encrypted Variables and signed Workload
Identities, but the key material is protected only by a AEAD encrypting the
KEK. Add support for Vault transit encryption and external KMS from major cloud
providers. The servers call out to the external service to decrypt each key in
the on-disk keystore.

Ref: https://hashicorp.atlassian.net/browse/NET-10334
Fixes: https://github.com/hashicorp/nomad/issues/14852
2024-07-18 09:42:28 -04:00
Juanadelacuesta
52ff858ff8 Add the missing v on the changelog call 2024-07-18 10:00:21 +02:00
Daniel Bennett
de10efa3fa e2e: hc-install consul-cni (#23612)
now that the version with tproxy CNI_ARGS is on releases.hashicorp.com
2024-07-17 14:26:40 -05:00
Daniel Bennett
afbd283c1b e2e: skip missing windows ami if windows clients=0 (#23610)
and tweak Makefile to generate a custom.tfvars
instead of specifying vars separately via CLI.
hoping this makes it a little more obvious
if there is no consul/nomad license.
2024-07-17 12:45:41 -05:00
Martina Santangelo
8cbd857ac9 cni: test Setup method (#23609)
Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
2024-07-17 13:02:23 -04:00
Juana De La Cuesta
3d93bd3778 Merge pull request #23607 from hashicorp/post-1.8.2-release
Post 1.8.2 release
2024-07-17 14:23:58 +02:00
Juanadelacuesta
656725a615 fix: updated ui assets 2024-07-17 13:59:51 +02:00
Juanadelacuesta
16677673e9 Merge release 1.8.2 files 2024-07-17 10:39:22 +02:00
Piotr Kazmierczak
f22ce921cd docker: adjust capabilities on Windows (#23599)
Adjusts Docker capabilities per OS, and checks for runtime on Windows.

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-07-17 09:01:45 +02:00
hc-github-team-nomad-core
c38b011c0c Prepare for next release 2024-07-17 00:00:37 +02:00
hc-github-team-nomad-core
6dc691da07 Generate files for 1.8.2 release 2024-07-17 00:00:36 +02:00
Martina Santangelo
bc81c85ec7 e2e: cni args tests (#23597)
Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
2024-07-15 17:08:50 -04:00
Tim Gross
d688b71f7f testing: fix serf join on keyring test and tighten serf config (#23594)
Nomad's default serf configuration has a full sync interval of 60s (the WAN
default configuration in the library). If tests need to join nodes and the
leader is not in the join set, the test can hang up to twice that interval
waiting for the new node to be seen by the leader and added to Raft.

This changeset includes the following tweaks to improve test timings:
* Ensure that nodes introduced later in the keyring replication test are joined
  to all peers. (Also updates the test to `shoenig/test`.)
* Update the `TestJoin` helper so that all servers passed are joined to the full
  set, instead of a set that's offset by 1, and use a single `Join` call for
  each server to reduce the number of messages sent.
* Reduce the `PushPullInterval` from 60s to 500ms in our unit test
  configuration, to force faster full syncs.
2024-07-15 14:17:46 -04:00
James Rasell
a65e5c126a docs: update quota docs and changelog to detail new cores feature. (#23592) 2024-07-15 10:07:34 +01:00
guifran001
1c44521543 client: Add a preferred address family option for network-interface (#23389)
to prefer ipv4 or ipv6 when deducing IP from network interface

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>
2024-07-12 15:30:38 -05:00
Martina Santangelo
b58b8114f9 fix cpucompat_default_test (#23575)
specifically !linux TestNUMA_topologyFromLegacy_plain
related: 7d73065066
2024-07-12 11:54:37 -05:00
Martina Santangelo
661011f5de cni: allow users to set CNI args in job spec (#23538) 2024-07-12 11:47:15 -04:00
Adrian Todorov
3f2729f7f5 remove mentions of old versions of Nomad in various docs (#23567) 2024-07-12 11:01:32 -04:00
Piotr Kazmierczak
fabae251c5 docs: correct deb and rpm registry installation instructions for podman driver (#23571) 2024-07-12 16:55:10 +02:00
Piotr Kazmierczak
b423276986 docs: update podman driver installation instructions (#23568)
Co-authored-by: Tim Gross <tgross@hashicorp.com>
2024-07-12 16:30:55 +02:00
Piotr Kazmierczak
ddbb307fa6 e2e: purge the job in the UI stop_proxy() script (#23565)
otherwise namespace deletion fails due to non-terminal allocations
2024-07-12 10:13:51 +02:00
Piotr Kazmierczak
fa8ffedd74 api: handle newlines in JobSubmission vars correctly (#23560)
Fixes a bug where variable values in job submissions that contained newlines
weren't encoded correctly, and thus jobs that contained them couldn't be
resumed once stopped via the UI.

Internal ref: https://hashicorp.atlassian.net/browse/NET-9966
2024-07-12 08:04:27 +02:00
Phil Renaud
75722ef93e Actions pass job id instead of job name (#23553) 2024-07-11 14:55:37 -04:00
Phil Renaud
370952e390 didUpdateAttributes manually called on modify (#23548) 2024-07-11 14:55:01 -04:00
Phil Renaud
0324e781d4 [ui] Global token CRUD in the web UI (#23506)
* First pass at global token creation and regional awareness at token fetch time

* Reset and refetch token when you switch region but stay in place

* Ugly and functional global token save

* Tests and log cleanup
2024-07-11 14:54:39 -04:00
Deniz Onur Duzgun
c82dd76a1b security: update tls cipher suites (#23551) 2024-07-11 14:01:45 -04:00
Adrian Todorov
6589d7130b docs: remove mentions of 'new in Nomad X version' where X is an older version (#23552) 2024-07-11 13:43:28 -04:00
Adrian Todorov
622995df42 fix link to point to Sentinel Enfrocement Levels (#23531) 2024-07-11 12:37:20 -04:00
Adrian Todorov
69d2eed808 Improve the restrict-images Sentinel policy template to only apply to Docker/Podman tasks and to handle version numbers starting with v (#23530) 2024-07-11 12:36:57 -04:00
Daniel Bennett
372dfd3816 msgpack: omit empty fields on NetworkResource (#23540)
the ratio of optimized/unoptimized log size in TestPlanNormalize
has been increased several times as people have added to various
structs and coincidentally bumped into the magic limit.

we encountered another such case when adding to NetworkResource,
but here we omitempty on the struct instead of bumping the limit
in the test.

this has the added benefit of reducing the serialized struct size!
which was the original intent behind this test in the first place :P
the actual value of the ratio is now 0.628... but here the
test value is only dropped down to 0.66 to leave some wiggle room.
2024-07-11 09:59:24 -05:00
James Rasell
f3de47e63d quota: Allow cores to be configured within an enterprise quota. (#23543) 2024-07-11 14:54:25 +01:00
dependabot[bot]
81a5eed99b chore(deps): bump github.com/container-storage-interface/spec (#23514)
Bumps [github.com/container-storage-interface/spec](https://github.com/container-storage-interface/spec) from 1.7.0 to 1.10.0.
- [Release notes](https://github.com/container-storage-interface/spec/releases)
- [Commits](https://github.com/container-storage-interface/spec/compare/v1.7.0...v1.10.0)

---
updated-dependencies:
- dependency-name: github.com/container-storage-interface/spec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 07:56:03 -05:00
dependabot[bot]
9381febedb chore(deps): bump github.com/klauspost/cpuid/v2 from 2.2.5 to 2.2.8 (#23513)
Bumps [github.com/klauspost/cpuid/v2](https://github.com/klauspost/cpuid) from 2.2.5 to 2.2.8.
- [Release notes](https://github.com/klauspost/cpuid/releases)
- [Changelog](https://github.com/klauspost/cpuid/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/cpuid/compare/v2.2.5...v2.2.8)

---
updated-dependencies:
- dependency-name: github.com/klauspost/cpuid/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 07:55:00 -05:00
dependabot[bot]
4ccc613165 build(deps): bump github.com/gorilla/handlers from 1.5.1 to 1.5.2 (#23356)
Bumps [github.com/gorilla/handlers](https://github.com/gorilla/handlers) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/gorilla/handlers/releases)
- [Commits](https://github.com/gorilla/handlers/compare/v1.5.1...v1.5.2)

---
updated-dependencies:
- dependency-name: github.com/gorilla/handlers
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 07:54:07 -05:00