Commit Graph

2854 Commits

Author SHA1 Message Date
Seth Hoenig
958942e340 Merge pull request #9608 from hashicorp/f-go-connlimit
Use go-connlimit to ratelimit with 429 responses
2020-12-10 11:05:07 -06:00
Seth Hoenig
26acbf750c client: fix plumbing of testing object into helper 2020-12-10 11:04:38 -06:00
Kris Hicks
7747124ef0 Apply some suggested fixes from staticcheck (#9598) 2020-12-10 07:29:18 -08:00
Seth Hoenig
5508e5d8cb client: fix https test cases in client rate limits 2020-12-10 09:20:28 -06:00
Ben Buzbee
1919c18b0f Use new go-connlimit with HTTP 429 response
This is essentially a port of Consul's similar fix
Changes are:
go get -u github.com/hashicorp/go-connlimit
go mod vendor
Use new HTTP429 handler

20d1ea7d2d
2020-12-09 17:57:16 -06:00
Mahmood Ali
6bfbadc199 actually honor var files (#9592)
Apparently, we missed passing VarFile argument, so var files were
ignored.

Fixes #9588
2020-12-09 14:12:48 -05:00
Kris Hicks
85ed8ddd4f Add gosimple linter (#9590) 2020-12-09 11:05:18 -08:00
Kris Hicks
071f4c7596 Add gocritic to golangci-lint config (#9556) 2020-12-08 12:47:04 -08:00
Dennis Schön
d294a7888b return 405 on non-GET requests to /v1/event/stream (fixes #9526) (#9564) 2020-12-08 13:09:20 -05:00
Dave May
d757507341 fix AgentHostRequest panic found in GH-9546 (#9554)
* debug: refactor nodeclass test
* debug: add case to track down SIGSEGV on client to server Agent.Host RPC
* verify server to avoid panic on AgentHostRequest RPC call, fixes GH-9546
* simplify Agent.Host RPC lookup logic
2020-12-07 17:34:40 -05:00
Dennis Schön
582d3b7092 use os.ErrDeadlineExceeded in tests 2020-12-07 10:40:28 -05:00
Dave May
d8070e99b1 nomad operator debug - add pprof duration / csi details (#9346)
* debug: add pprof duration CLI argument
* debug: add CSI plugin details
* update help text with ACL requirements
* debug: provide ACL hints upon permission failures
* debug: only write file when pprof retrieve is successful
* debug: add helper function to clean bad characters from dynamic filenames
* debug: ensure files are unable to escape the capture directory
2020-12-01 12:36:05 -05:00
Drew Bailey
61ce743228 Event Stream: Track ACL changes, unsubscribe on invalidating changes (#9447)
* upsertaclpolicies

* delete acl policies msgtype

* upsert acl policies msgtype

* delete acl tokens msgtype

* acl bootstrap msgtype

wip unsubscribe on token delete

test that subscriptions are closed after an ACL token has been deleted

Start writing policyupdated test

* update test to use before/after policy

* add SubscribeWithACLCheck to run acl checks on subscribe

* update rpc endpoint to use broker acl check

* Add and use subscriptions.closeSubscriptionFunc

This fixes the issue of not being able to defer unlocking the mutex on
the event broker in the for loop.

handle acl policy updates

* rpc endpoint test for terminating acl change

* add comments

Co-authored-by: Kris Hicks <khicks@hashicorp.com>
2020-12-01 11:11:34 -05:00
Drew Bailey
bf225f7179 Remove Managed Sinks from Nomad (#9470)
* Remove Managed Sinks from Nomad

Managed Sinks were a beta feature in Nomad 1.0-beta2. During the beta
period it was determined that this was not a scalable approach to
support community and third party sinks.

* update comment

* changelog
2020-11-30 14:00:31 -05:00
Seth Hoenig
f5806ee0b0 Merge pull request #9472 from hashicorp/f-connect-upstream-datacenter
consul/connect: enable setting datacenter in upstream
2020-11-30 12:29:05 -06:00
Seth Hoenig
ce32e31195 consul/connect: enable setting datacenter in connect upstream
Before, upstreams could only be defined using the default datacenter.
Now, the `datacenter` field can be set in a connect upstream definition,
informing consul of the desire for an instance of the upstream service
in the specified datacenter. The field is optional and continues to
default to the local datacenter.

Closes #8964
2020-11-30 10:38:30 -06:00
Chris Piwarski
57fe67ce53 command: Fix node help error 2020-11-30 11:12:06 -05:00
Tim Gross
8351c3f9d6 csi/api: populate ReadAllocs/WriteAllocs fields (#9377)
The API is missing values for `ReadAllocs` and `WriteAllocs` fields, resulting
in allocation claims not being populated in the web UI. These fields mirror
the fields in `nomad/structs.CSIVolume`. Returning a separate list of stubs
for read and write would be ideal, but this can't be done without either
bloating the API response with repeated full `Allocation` data, or causing a
panic in previous versions of the CLI.

The `nomad/structs` fields are persisted with nil values and are populated
during RPC, so we'll do the same in the HTTP API and populate the `ReadAllocs`
and `WriteAllocs` fields with a map of allocation IDs, but with null
values. The web UI will then create its `ReadAllocations` and
`WriteAllocations` fields by mapping from those IDs to the values in
`Allocations`, instead of flattening the map into a list.
2020-11-25 16:44:06 -05:00
Seth Hoenig
d5a422e320 api: safely access legacy MBits field 2020-11-23 10:36:10 -06:00
Nick Ethier
f8aabc8bd2 command: use correct port mapping syntax in examples 2020-11-23 10:25:30 -06:00
Nick Ethier
1c14f69853 command: remove task network usage from init examples 2020-11-23 10:25:11 -06:00
Nick Ethier
b12d8a29f1 mock: add default host network 2020-11-23 10:11:00 -06:00
Tim Gross
1f8a7fc047 docs: add missing command documentation (#9415)
* `nomad operator keyring` was missing the general options section
* `nomad operator metrics` was missing a page in the docs entirely

Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>
2020-11-23 08:10:10 -05:00
Tim Gross
8a66f11bb3 docs: describe required ACLs for all commands 2020-11-20 13:38:29 -05:00
Tim Gross
89f4f51746 command: remove -namespace from help options when not applicable 2020-11-19 16:28:39 -05:00
Tim Gross
dd00f2ab23 docs/help: -no-color does not apply to alloc logs content
The `nomad alloc logs` command does not remove terminal escape sequences for
color from the log outputs of a task. Clarify that the standard `-no-color`
flag, which does apply to Nomad's error responses from `nomad alloc logs`,
does not apply to the log output.
2020-11-19 15:29:12 -05:00
Chris Baker
7a6db5f483 Merge pull request #9368 from hashicorp/b-9367-duped-ports
fix duplication of ports in `AllocatedResources`
2020-11-16 11:41:34 -06:00
Tim Gross
78afe18966 csi: alloc status -verbose should query volume request 'source' (#9354)
The `nomad alloc status -verbose` command returns a 404 from CSI volumes
because the volume mount block in the task points back to the
`job.group.volume` block. So using the `Name` field to query is the "name" as
seen in the jobspec, and not the name of the volume that we need for querying.

Show both the job-specific name and the volume ID in the resulting output,
which clarifies the difference between the two fields and is more consistent
with the web UI.
2020-11-16 08:28:52 -05:00
Chris Baker
2a409de5d8 updated alloc_endpoint to mutate a copy of the returned allocation, instead of the instance in the state store 2020-11-15 17:52:50 +00:00
Chris Baker
af5478ccc2 documenting test for #9367 2020-11-15 17:47:50 +00:00
Seth Hoenig
459112b41d Merge pull request #9352 from hashicorp/f-artifact-headers
jobspec: add support for headers in artifact stanza
2020-11-13 14:04:27 -06:00
Seth Hoenig
6c7578636c jobspec: add support for headers in artifact stanza
This PR adds the ability to set HTTP headers when downloading
an artifact from an `http` or `https` resource.

The implementation in `go-getter` is such that a new `HTTPGetter`
must be created for each artifact that sets headers (as opposed
to conveniently setting headers per-request). This PR maintains
the memoization of the default Getter objects, creating new ones
only for artifacts where headers are set.

Closes #9306
2020-11-13 12:03:54 -06:00
Dave May
205b0e7cae nomad operator debug - add client node filtering arguments (#9331)
* operator debug - add client node filtering arguments

* add WaitForClient helper function

* use RPC in WaitForClient to avoid unnecessary imports

* guard against nil values

* move initialization up and shorten test duration

* cleanup nodeLookupFailCount logic

* only display max node notice if we actually tried to capture nodes
2020-11-12 11:25:28 -05:00
Seth Hoenig
685caee3df consul: prevent re-registration churn by correctly comparing sidecar tags
Previously, connect sidecars would be re-registered with consul every cycle
of Nomad's reconciliation loop around Consul service registrations. This is
because part of the comparison used `reflect.DeepEqual` on []string objects,
which returns false when one object is `[]string{}` and the other is `[]string{}(nil)`.

Unforunately, this was always the case, and every Connect sidecar service
would be re-registered on every iteration, which happens every 30 seconds.
2020-11-11 18:01:17 -06:00
Chris Baker
93e1c7f6ce Merge pull request #9327 from hashicorp/b-fix-rec-cli
recs cli: fixed bad composition
2020-11-11 15:54:14 -06:00
Nick Ethier
d947690c51 structs: canonicalize allocatedtaskresources to populate shared ports (#9309) 2020-11-11 16:21:47 -05:00
Chris Baker
3d159df7b8 recs cli: fixed bad composition 2020-11-11 20:52:57 +00:00
Tim Gross
0ed0b945c9 csi: Postrun hook should not change mode (#9323)
The unpublish workflow requires that we know the mode (RW vs RO) if we want to
unpublish the node. Update the hook and the Unpublish RPC so that we mark the
claim for release in a new state but leave the mode alone. This fixes a bug
where RO claims were failing node unpublish.

The core job GC doesn't know the mode, but we don't need it for that workflow,
so add a mode specifically for GC; the volumewatcher uses this as a sentinel
to check whether claims (with their specific RW vs RO modes) need to be claimed.
2020-11-11 13:06:30 -05:00
Chris Baker
8b0d273dd9 cli: updated recommendation commands and test to remove duplication of autocompletion code 2020-11-11 11:39:26 +00:00
Chris Baker
316ca44667 auto-complete for recommendations CLI, plus OSS components of recommendations prefix search 2020-11-11 11:13:43 +00:00
Mahmood Ali
73df83fffd Custom message when job file is HCL2 incompatible
Use a custom message when the job file is a valid HCL1 but no longer
valid under HCL 2 syntax.
2020-11-10 14:56:20 -05:00
Mahmood Ali
11b968c7cb Merge pull request #9298 from hashicorp/f-hcl2-localsvars
HCL2: Variables and Locals
2020-11-09 16:44:37 -05:00
Luiz Aoqui
6881346f12 Merge pull request #9296 from hashicorp/b-remove-namespace-from-scale-request
Remove Namespace field from JobScaleRequest
2020-11-09 15:13:33 -05:00
Mahmood Ali
dda76b2e03 Start accepting input variable files as CLI arguments 2020-11-09 15:02:58 -05:00
Mahmood Ali
5a784f43a3 Start using the new jobspec2 API 2020-11-09 15:01:31 -05:00
Luiz Aoqui
b9991a3f2b remove Namespace field from JobScaleRequest 2020-11-09 13:02:05 -05:00
James Rasell
3fe55a4f2d Merge pull request #9272 from hashicorp/f-recommendation-cli
cli: add recommendation commands.
2020-11-06 16:29:57 +01:00
James Rasell
0cd4d75aab cli: add recommendation commands.
Adds new CLI commands for applying, dismissing and detailing Nomad
recommendations under a new top level recommendation command.
2020-11-06 11:16:24 +01:00
Nick Ethier
60838c94f8 ar/groupservice: remove drivernetwork (#9233)
* ar/groupservice: remove drivernetwork

* consul: allow host address_mode to accept raw port numbers

* consul: fix logic for blank address
2020-11-05 15:00:22 -05:00
Charlie Voiselle
6c651f193d Merge pull request #9255 from hashicorp/d-missing-example-comma
Add missing comma in help-text example.
2020-11-04 09:51:09 -05:00