Commit Graph

21557 Commits

Author SHA1 Message Date
Kent 'picat' Gruber
fc3ff8ee13 Update deployment status doc example output 2021-07-16 09:58:58 -04:00
Kent 'picat' Gruber
4bcc72da1a Handle DeploymentStatusFailed unless hasAutoRevert 2021-07-15 17:06:13 -04:00
Kent 'picat' Gruber
440d556fdb Handle successful/canceled/blocked deployments in CLI output
Otherwise the spinner would just end, which felt a bit awkward.

I wanted to see a  "✓" to know that everything was ok, and a "!" (maybe something else?) if something went wrong.
2021-07-09 19:27:55 -04:00
Zachary Shilton
c4e0355775 docs: remove unnecessary redirect (#10802) 2021-07-09 09:44:34 -04:00
Mike Nomitch
9235c790f0 [docs] Adds federation caveat to upgrade guide (#10847) 2021-07-09 09:42:17 -04:00
Mahmood Ali
8d1a6ea1d5 e2e: use -detach mode when registering jobs with cli (#10877)
Pick up 15d39f0dee but for RegisterFromJobspec:

>  This PR changes the e2e helper thingy to set -detach option
>  when registering a job with the CLI instead of the API. This is
>  necessary for jobs which never become healthy, as the deployment
>  never finishes for failing jobs and the command never returns,
>  causing the test to timeout after 10 minutes.

This case occurs in TestVaultSecrets
2021-07-09 09:25:44 -04:00
Shishir
24f88398c3 Containerd task driver options. (#10878)
- Auth support via driver config
- pid_mode
- shm_size

Signed-off-by: Shishir Mahajan <smahajan@roblox.com>
2021-07-09 08:17:42 -04:00
Tim Gross
d0fb5c8c2a client: interpolate meta blocks with task environment (#10876)
Adds missing interpolation step to the `meta` blocks when building the task
environment. Also fixes incorrect parameter order in the test assertion and
adds diagnostics to the test.
2021-07-08 16:03:15 -04:00
Seth Hoenig
0e49f0d20a Merge pull request #10873 from hashicorp/b-cc-rm-sidecar-first
consul/connect: remove sidecar proxy before removing parent service
2021-07-08 14:29:06 -05:00
Seth Hoenig
8ad8e1095a consul/connect: remove sidecar proxy before removing parent service
This PR will have Nomad de-register a sidecar proxy service before
attempting to de-register the parent service. Otherwise, Consul will
emit a warning and an error.

Fixes #10845
2021-07-08 13:30:19 -05:00
Seth Hoenig
72f431f4c4 Merge pull request #10872 from hashicorp/b-cc-regex-checkids
consul/connect: Avoid assumption of parent service when filtering connect proxies
2021-07-08 13:29:40 -05:00
Seth Hoenig
82f424585a consul/connect: improve regex from CR suggestions 2021-07-08 13:05:05 -05:00
Seth Hoenig
10f8b4e3e1 Merge pull request #10874 from hashicorp/b-fix-logline
client: fix logline in group shutdown hook
2021-07-08 12:59:00 -05:00
Seth Hoenig
d783dc940a client: fix logline in group shutdown hook
Fixes #10844
2021-07-08 11:14:37 -05:00
Seth Hoenig
86283ad2dc consul/connect: Avoid assumption of parent service when filtering connect proxies
This PR uses regex-based matching for sidecar proxy services and checks when syncing
with Consul. Previously we would check if the parent of the sidecar was still being
tracked in Nomad. This is a false invariant - one which we must not depend when we
make #10845 work.

Fixes #10843
2021-07-08 09:43:41 -05:00
Mahmood Ali
0f0bcdb6ed Merge pull request #10806 from hashicorp/munda/idempotent-job-dispatch
Enforce idempotency of dispatched jobs using token on dispatch request
2021-07-08 10:23:31 -04:00
Tim Gross
b397edff4e cni: respect default cni_config_dir and cni_path (#10870)
The default agent configuration values were not set, which meant they were not
being set in the client configuration and this results in fingerprints failing
unless the values were set explicitly.
2021-07-08 09:56:57 -04:00
Tim Gross
3f6ece54d9 service: remove duplicate name check during validation (#10868)
When a task group with `service` block(s) is validated, we validate that there
are no duplicates, but this validation doesn't have access to the task environment
because it hasn't been created yet. Services and checks with interpolation can
be flagged incorrectly as conflicting. Name conflicts in services are not
actually an error in Consul and users have reported wanting to use the same
service name for task groups differentiated by tags.
2021-07-08 09:43:38 -04:00
Tim Gross
d5f4fc7d6f testing: prevent panic when job status output changes (#10869)
The `command/TestJobStatusCommand_Run` test assumes that it gets back running
allocations and will panic the test runner rather than failing.
2021-07-08 09:25:44 -04:00
Alex Munda
d77329a6d2 Sync vendored api 2021-07-07 19:53:46 -05:00
Alex Munda
16d43ae6a5 Changelog entry for dispatch idempotency token 2021-07-07 19:48:59 -05:00
Alex Munda
2bd2f586f4 Set/parse idempotency_token query param 2021-07-07 16:26:55 -05:00
Alex Munda
3a8febe5bf Update idempotency comment to reflect all jobs
Co-authored-by: Mahmood Ali <mahmood@hashicorp.com>
2021-07-07 15:54:56 -05:00
Seth Hoenig
60a92ffee5 Merge pull request #10865 from hashicorp/b-deregister-noops
consul: avoid extra sync operations when no action required
2021-07-07 13:42:46 -05:00
Isabel Suchanek
3117d6c78e cli: add -task flag to alloc signal, restart (#10859)
Alloc exec only works when task is passed as a flag and not an arg.
Alloc logs currently accepts either, but alloc signal and restart only
accept task as an arg. This adds -task as a flag to the other alloc
commands to make the cli UX consistent. If task is passed as a flag and
an arg, it ignores the arg.
2021-07-07 09:58:16 -07:00
Seth Hoenig
421a6a8a7e consul: avoid extra sync operations when no action required
This PR makes it so the Consul sync logic will ignore operations that
do not specify an action to take (i.e. [de-]register [services|checks]).

Ideally such noops would be discarded at the callsites (i.e. users
of [Create|Update|Remove]Workload], but we can also be defensive
at the commit point.

Also adds 2 trace logging statements which are helpful for diagnosing
sync operations with Consul - when they happen and why.

Fixes #10797
2021-07-07 11:24:56 -05:00
Seth Hoenig
5e0ffb94b7 Merge pull request #10857 from hashicorp/b-rm-canarys
consul: avoid triggering unnecessary sync when removing workload
2021-07-07 09:47:15 -05:00
James Rasell
aa8940fb42 Merge pull request #10861 from hashicorp/f-gh-10860
api: Added `NewSystemJob` job creation helper function.
2021-07-07 16:17:15 +02:00
Mahmood Ali
712ad497b1 Merge pull request #10743 from holtwilkins/parse-term-gws
Parse term gws
2021-07-07 10:13:37 -04:00
Tim Gross
550fca9f83 csi: account for nil volume_mount in API-to-structs conversion (#10855)
Fix a nil pointer in the API struct to `nomad/structs` conversion when a
`volume_mount` block is empty.
2021-07-07 08:06:39 -04:00
James Rasell
01a551faaf changelog: add entry for #10861 2021-07-07 11:03:39 +02:00
James Rasell
8fc2d79245 api: Added NewSystemJob job creation helper function. 2021-07-07 11:03:20 +02:00
Seth Hoenig
b937e7baf4 consul: avoid triggering unnecessary sync when removing workload
There are bits of logic in callers of RemoveWorkload on group/task
cleanup hooks which call RemoveWorkload with the "Canary" version
of the workload, in case the alloc is marked as a Canary. This logic
triggers an extra sync with Consul, and also doesn't do the intended
behavior - for which no special casing is necessary anyway. When the
workload is marked for removal, all associated services and checks
will be removed regardless of the Canary status, because the service
and check IDs do not incorporate the canary-ness in the first place.

The only place where canary-ness matters is when updating a workload,
where we need to compute the hash of the services and checks to determine
whether they have been modified, the Canary flag of which is a part of
that.

Fixes #10842
2021-07-06 14:08:42 -05:00
Mahmood Ali
18d359f71d Adopt go-changelog in Nomad (#10825)
Adopts [`go-changelog`](https://github.com/hashicorp/go-changelog) for managing Nomad's changelog. `go-changelog` is becoming the HashiCorp defacto standard tool for managing changelog, e.g. [Consul](https://github.com/hashicorp/consul/pull/8387), [Vault](https://github.com/hashicorp/vault/pull/10363), [Waypoint](https://github.com/hashicorp/waypoint/pull/1179). [Consul](https://github.com/hashicorp/consul/pull/8387) seems to be the first product to adopt it, and its PR has the most context - though I've updated `.changelog/README.md` with the relevant info here.

## Changes to developers workflow

When opening PRs, developers should add a changelog entry in `.changelog/<PR#>.txt`. Check [`.changelog/README.md`](https://github.com/hashicorp/nomad/blob/docs-adopt-gochangelog/.changelog/README.md#developer-guide). 

For the WIP release, entries can be amended even after the PR merged, and new files may be added post-hoc (e.g. during transition period, missed accidentally, community PRs, etc).

### Transitioning

Pending PRs can start including the changelog entry files immediately.

For 1.1.3/1.0.9 cycle, the release coordinator should create the entries for any PR that gets merged without a changelog entry file. They should also move any 1.1.3 entry in CHANGELOG.md to a changelog entry file, as this PR done for GH-10818.

## Changes to release process

Before cutting a release, release coordinator should update the changelog by inserting the output of `make changelog` to CHANGELOG.md with appropriate headers. See [`.changelog/README.md`](https://github.com/hashicorp/nomad/blob/docs-adopt-gochangelog/.changelog/README.md#how-to-generate-changelog-entries-for-release) for more details.


## Details

go-changelog is a basic templating engine for maintaining changelog in HashiCorp environment.

It expects the changelog entries as files indexed by their PR number. The CLI generates the changelog section for a release by comparing two git references (e.g. `HEAD` and the latest release, e.g. `v1.1.2`), and still requires manual process for updating CHANGELOG.md and final formatting.

The approach has many nice advantages:
* Avoids changelog related merge conflicts: Each PR touches different file!
* Copes with amendments and post-PR updates: Just add or update a changelog entry file using the original PR numbers.
* Addresses the release backporting scenario: Cherry-picking PRs will cherry-pick the relevant changelog entry automatically!
* Only relies on data available through `git` - no reliance on GitHub metadata or require GitHub credentials

The approach has few downsides though:
* CHANGELOG.md going stale during development and must be updated manually before cutting the release
  * Repository watchers can no longer glance at the CHANGELOG.md to see upcoming changes
  * We can periodically update the file, but `go-changelog` tool does not aid with that
* `go-changelog` tool does not offer good error reporting. If an entry is has an invalid tag (e.g. uses `release-note:bugfix` instead of `release-note:bug`), the entry will be dropped silently
  * We should update go-changelog to warn against unexpected entry tags
  * TODO: Meanwhile, PR reviewers and release coordinators should watch out

## Potential follow ups

We should follow up with CI checks to ensure PR changes include a warning. I've opted not to include that now. We still make many non-changelog-worth PRs for website/docs, for large features that get merged in multiple small PRs. I did not want to include a check that fails often.

Also, we should follow up to have `go-changelog` emit better warnings on unexpected tag.
2021-07-06 10:46:53 -04:00
James Rasell
062f441fe7 Merge pull request #10853 from shantanugadgil/patch-4
constraint on the Docker driver missing 'attr'
2021-07-06 12:19:08 +02:00
Shantanu Gadgil
751f66f3a6 constraint on the Docker driver missing 'attr'
Between this page and https://www.nomadproject.io/docs/runtime/interpolation
I realized that the syntax on the Docker page was missing the word `attr`.
2021-07-06 08:48:04 +05:30
Alex Munda
6a1a200925 Match idempotency key on all child jobs and return existing job when idempotency keys match. 2021-07-02 14:08:46 -05:00
Alex Munda
4448cff8a3 Move idempotency check closer to validate. Log error. 2021-07-02 10:58:42 -05:00
Grant Griffiths
4e9ed5f882 CSI: Snapshot volume create should use vol.Secrets (#10840)
Signed-off-by: Grant Griffiths <ggriffiths@purestorage.com>
2021-07-02 08:28:22 -04:00
Alex Munda
aa3512d11c Update tests after moving idempotency token to WriteOptions 2021-07-01 08:48:57 -05:00
Alex Munda
63d8e92e90 Move idempotency token to write options. Remove DispatchIdempotent 2021-06-30 15:10:48 -05:00
Alex Munda
d83ff8b8c9 Update comment about idempotency check 2021-06-30 12:30:44 -05:00
Alex Munda
850204870f Make idempotency error user friendly
Co-authored-by: Tim Gross <tgross@hashicorp.com>
2021-06-30 12:26:33 -05:00
Tim Gross
a66034bb3a docker: move host path for hosts file mount to alloc dir (#10823)
In Nomad 1.1.1 we generate a hosts file based on the Nomad-owned network
namespace, rather than using the default hosts file from the pause
container. This hosts file should be shared between tasks in the same
allocation so that tasks can update the file and have the results propagated
between tasks.
2021-06-30 11:10:04 -04:00
James Rasell
ff8e98e612 Merge pull request #10829 from hashicorp/gh-10820-docs
changelog: add entry for #10822
2021-06-30 08:37:02 +02:00
James Rasell
3b735129ef Merge pull request #10831 from jcalonso/patch-1
[docs] Fix missing opening quote
2021-06-30 08:36:39 +02:00
Holt Wilkins
898c6c59a6 Enable parsing of terminating gateways 2021-06-30 05:34:16 +00:00
Juan Carlos Alonso
acd13314ed Fix missing opening quote 2021-06-30 12:33:34 +08:00
Alex Munda
f607c35421 Add idempotency token to dispatch request instead of special meta key 2021-06-29 15:59:23 -05:00
Seth Hoenig
b5c3f4e9df Merge pull request #10805 from hashicorp/b-cn-tls-env
consul/connect: automatically set consul tls sni name for connect native tasks
2021-06-29 14:10:21 -05:00