Commit Graph

16946 Commits

Author SHA1 Message Date
Seth Hoenig
d252bb4e80 e2e: do not use eventually when waiting for allocs
This test is causing panics. Unlike the other similar tests, this
one is using require.Eventually which is doing something bad, and
this change replaces it with a for-loop like the other tests.

Failure:

=== RUN   TestE2E/Connect
=== RUN   TestE2E/Connect/*connect.ConnectE2ETest
=== RUN   TestE2E/Connect/*connect.ConnectE2ETest/TestConnectDemo
=== RUN   TestE2E/Connect/*connect.ConnectE2ETest/TestMultiServiceConnect
=== RUN   TestE2E/Connect/*connect.ConnectClientStateE2ETest
panic: Fail in goroutine after TestE2E/Connect/*connect.ConnectE2ETest has completed

goroutine 38 [running]:
testing.(*common).Fail(0xc000656500)
	/opt/google/go/src/testing/testing.go:565 +0x11e
testing.(*common).Fail(0xc000656100)
	/opt/google/go/src/testing/testing.go:559 +0x96
testing.(*common).FailNow(0xc000656100)
	/opt/google/go/src/testing/testing.go:587 +0x2b
testing.(*common).Fatalf(0xc000656100, 0x1512f90, 0x10, 0xc000675f88, 0x1, 0x1)
	/opt/google/go/src/testing/testing.go:672 +0x91
github.com/hashicorp/nomad/e2e/connect.(*ConnectE2ETest).TestMultiServiceConnect.func1(0x0)
	/home/shoenig/go/src/github.com/hashicorp/nomad/e2e/connect/multi_service.go:72 +0x296
github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert.Eventually.func1(0xc0004962a0, 0xc0002338f0)
	/home/shoenig/go/src/github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert/assertions.go:1494 +0x27
created by github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert.Eventually
	/home/shoenig/go/src/github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert/assertions.go:1493 +0x272
FAIL	github.com/hashicorp/nomad/e2e	21.427s
2020-01-31 19:05:47 -06:00
Seth Hoenig
1c9500ab27 e2e: remove forgotten unused field from new struct 2020-01-31 19:05:41 -06:00
Seth Hoenig
6d99dd4841 e2e: e2e test for connect with consul acls
Provide script for managing Consul ACLs on a TF provisioned cluster for
e2e testing. Script can be used to 'enable' or 'disable' Consul ACLs,
and automatically takes care of the bootstrapping process if necessary.

The bootstrapping process takes a long time, so we may need to
extend the overall e2e timeout (20 minutes seems fine).

Introduces basic tests for Consul Connect with ACLs.
2020-01-31 19:05:36 -06:00
Seth Hoenig
0f285b840e tests: skip some SIDS hook tests if running tests as root 2020-01-31 19:05:32 -06:00
Seth Hoenig
08951ac759 client: additional test cases around failures in SIDS hook 2020-01-31 19:05:27 -06:00
Seth Hoenig
91c7dbaa8d client: PR cleanup - improved logging around kill task in SIDS hook 2020-01-31 19:05:23 -06:00
Seth Hoenig
f8949dde35 client: PR cleanup - shadow context variable 2020-01-31 19:05:19 -06:00
Seth Hoenig
0589b656b7 nomad: make TaskGroup.UsesConnect helper a public helper 2020-01-31 19:05:11 -06:00
Seth Hoenig
6152ded0f4 nomad: fix leftover missed refactoring in consul policy checking 2020-01-31 19:05:06 -06:00
Seth Hoenig
40de85867d client: manage TR kill from parent on SI token derivation failure
Re-orient the management of the tr.kill to happen in the parent of
the spawned goroutine that is doing the actual token derivation. This
makes the code a little more straightforward, making it easier to
reason about not leaking the worker goroutine.
2020-01-31 19:05:02 -06:00
Seth Hoenig
1fca495a85 client: set context timeout around SI token derivation
The derivation of an SI token needs to be safegaurded by a context
timeout, otherwise an unresponsive Consul could cause the siHook
to block forever on Prestart.
2020-01-31 19:04:56 -06:00
Seth Hoenig
bbedeb670d nomad,client: apply more comment/style PR tweaks 2020-01-31 19:04:52 -06:00
Seth Hoenig
cc7b768907 nomad,client: apply smaller PR suggestions
Apply smaller suggestions like doc strings, variable names, etc.

Co-Authored-By: Nick Ethier <nethier@hashicorp.com>
Co-Authored-By: Michael Schurter <mschurter@hashicorp.com>
2020-01-31 19:04:40 -06:00
Seth Hoenig
d24d470775 comments: cleanup some leftover debug comments and such 2020-01-31 19:04:35 -06:00
Seth Hoenig
e825a0f769 client: skip task SI token file load failure if testing as root
The TestEnvoyBootstrapHook_maybeLoadSIToken test case only works when
running as a non-priveleged user, since it deliberately tries to read
an un-readable file to simulate a failure loading the SI token file.
2020-01-31 19:04:30 -06:00
Seth Hoenig
4b4dfacda5 client: remove unused indirection for referencing consul executable
Was thinking about using the testing pattern where you create executable
shell scripts as test resources which "mock" the process a bit of code
is meant to fork+exec. Turns out that wasn't really necessary in this case.
2020-01-31 19:04:25 -06:00
Seth Hoenig
ead935d12c agent: re-enable the server in dev mode 2020-01-31 19:04:19 -06:00
Seth Hoenig
9f48d83378 nomad: handle SI token revocations concurrently
Be able to revoke SI token accessors concurrently, and also
ratelimit the requests being made to Consul for the various
ACL API uses.
2020-01-31 19:04:14 -06:00
Seth Hoenig
d85cccc8d0 nomad: fixup token policy validation 2020-01-31 19:04:08 -06:00
Seth Hoenig
6bc6a52f99 client: enable envoy bootstrap hook to set SI token
When creating the envoy bootstrap configuration, we should append
the "-token=<token>" argument in the case where the sidsHook placed
the token in the secrets directory.
2020-01-31 19:04:01 -06:00
Seth Hoenig
674ccaa122 nomad: proxy requests for Service Identity tokens between Clients and Consul
Nomad jobs may be configured with a TaskGroup which contains a Service
definition that is Consul Connect enabled. These service definitions end
up establishing a Consul Connect Proxy Task (e.g. envoy, by default). In
the case where Consul ACLs are enabled, a Service Identity token is required
for these tasks to run & connect, etc. This changeset enables the Nomad Server
to recieve RPC requests for the derivation of SI tokens on behalf of instances
of Consul Connect using Tasks. Those tokens are then relayed back to the
requesting Client, which then injects the tokens in the secrets directory of
the Task.
2020-01-31 19:03:53 -06:00
Seth Hoenig
f8666bb1f9 client: enable nomad client to request and set SI tokens for tasks
When a job is configured with Consul Connect aware tasks (i.e. sidecar),
the Nomad Client should be able to request from Consul (through Nomad Server)
Service Identity tokens specific to those tasks.
2020-01-31 19:03:38 -06:00
Seth Hoenig
560ce96b90 nomad: ensure a unique ClusterID exists when leader (gh-6702)
Enable any Server to lookup the unique ClusterID. If one has not been
generated, and this node is the leader, generate a UUID and attempt to
apply it through raft.

The value is not yet used anywhere in this changeset, but is a prerequisite
for gh-6701.
2020-01-31 19:03:26 -06:00
Seth Hoenig
0040c75e8e command, docs: create and document consul token configuration for connect acls (gh-6716)
This change provides an initial pass at setting up the configuration necessary to
enable use of Connect with Consul ACLs. Operators will be able to pass in a Consul
Token through `-consul-token` or `$CONSUL_TOKEN` in the `job run` and `job revert`
commands (similar to Vault tokens).

These values are not actually used yet in this changeset.
2020-01-31 19:02:53 -06:00
Mahmood Ali
74e10b1062 Merge pull request #7043 from hashicorp/b-collection-interval
Pass stats interval colleciton to executor
2020-01-31 15:03:30 -05:00
Mahmood Ali
aec066c997 changelog 2020-01-31 14:22:08 -05:00
Mahmood Ali
87c0c92ac7 Pass stats interval colleciton to executor
This fixes a bug where executor based drivers emit stats every second,
regardless of user configuration.

When serializing the Stats request across grpc, the nomad agent dropped
the Interval value, and then executor uses 1s as a default value.
2020-01-31 14:17:15 -05:00
Michael Lange
6e61dcf617 Merge pull request #7028 from hashicorp/f-ui/node-drain-disable
UI: Disable client write actions when ACL token only allows client read
2020-01-31 10:20:06 -08:00
Mahmood Ali
b57d6064a8 Merge pull request #7041 from tiangolo/patch-1
Use secret ID for NOMAD_TOKEN
2020-01-31 13:00:40 -05:00
Sebastián Ramírez
ac696a4981 Use secret ID for NOMAD_TOKEN
Use secret ID for NOMAD_TOKEN as the accessor ID doesn't seem to work.

I tried with a local micro cluster following the tutorials, and if I do:

```console
$ export NOMAD_TOKEN=85310d07-9afa-ef53-0933-0c043cd673c7
```

Using the accessor ID as in this example, I get an error:

```
Error querying jobs: Unexpected response code: 403 (ACL token not found)
```

But when using the secret ID in that env var it seems to work correctly.
2020-01-31 18:57:16 +01:00
Michael Lange
234b7efee8 Acceptance test for disabled node write controls 2020-01-31 09:41:37 -08:00
Michael Lange
f5662817ff Account for disabled ACLs in ability tests 2020-01-31 09:41:36 -08:00
Michael Lange
35d7744ba7 Handle the case where ACLs aren't enabled in abilities 2020-01-31 09:41:36 -08:00
Michael Lange
e7067f0cc9 Fix token referencing from the token controller, as well as resetting 2020-01-31 09:41:35 -08:00
Michael Lange
692a88712d Add an explanatory tooltip to the unauthorized node drain popover 2020-01-31 09:41:33 -08:00
Michael Lange
478e6d831f Update disabled 'Run Job' button to use standard disabled style 2020-01-31 09:41:32 -08:00
Mahmood Ali
788f64e031 Merge pull request #7010 from hashicorp/doc-bulk-20200129
Docs and Changelog catch up
2020-01-31 10:51:07 -05:00
Tim Gross
a89beee256 hclfmt a test jobspec (#7011) 2020-01-31 08:04:03 -05:00
Michael Lange
c6a677dc4a Disable options for popover and drain-popover 2020-01-30 21:29:29 -08:00
Michael Lange
29c3cdaeb6 Disabled button styles 2020-01-30 21:29:28 -08:00
Michael Lange
35a95177c6 New disabled buttons story 2020-01-30 21:29:26 -08:00
Michael Lange
f51c0c4400 Refetch all ACL things when the token changes 2020-01-30 21:29:24 -08:00
Michael Lange
fa3a4e4074 Enable the eligibility toggle conditionally based on acls 2020-01-30 21:29:22 -08:00
Michael Lange
fbb137bdb2 Refactor ability tests to use a setup hook for ability lookup 2020-01-30 21:29:21 -08:00
Michael Lange
88d646b48d Add an ability for client permissions 2020-01-30 21:29:19 -08:00
Michael Schurter
b6933221bb Merge pull request #7026 from hashicorp/post-0103
Post 0103
2020-01-30 15:25:26 -08:00
Michael Schurter
0cd7eea939 docs: add v0.10.3 release to changelog 2020-01-30 15:24:33 -08:00
Michael Schurter
7c53d0e2d7 docs: bump 0.10.2 -> 0.10.3 2020-01-30 15:22:59 -08:00
Michael Schurter
f5441e69d4 Merge pull request #7023 from hashicorp/b-tls-validation
Validate role and region for mTLS
2020-01-30 11:05:37 -08:00
Michael Schurter
cb9274acd4 Merge branch 'master' into b-tls-validation 2020-01-30 11:05:15 -08:00