Commit Graph

16973 Commits

Author SHA1 Message Date
Drew Bailey
39a6c6374a Fix panic when monitoring a local client node
Fixes a panic when accessing a.agent.Server() when agent is a client
instead. This pr removes a redundant ACL check since ACLs are validated
at the RPC layer. It also nil checks the agent server and uses Client()
when appropriate.
2020-02-03 13:20:04 -05:00
Mahmood Ali
e8136c0c66 Merge pull request #7045 from hashicorp/b-rpc-fixes
Some fixes to connection pooling
2020-02-03 13:10:15 -05:00
Seth Hoenig
7f43161717 Merge pull request #7054 from hashicorp/f-remove-leftover-debug-line
e2e: remove leftover e2e debug println
2020-02-03 12:02:19 -06:00
Mahmood Ali
3bfc7d125d pool: Clear connection before releasing
This to be consistent with other connection clean up handler as well as consul's https://github.com/hashicorp/consul/blob/v1.6.3/agent/pool/pool.go#L468-L479 .
2020-02-03 12:41:11 -05:00
Seth Hoenig
9ccaa92ba1 e2e: remove leftover debug println statement 2020-02-03 11:15:38 -06:00
Mahmood Ali
445d0199b7 Merge pull request #7051 from hashicorp/b-copy-jobs-oss
sentinel: copy jobs to prevent mutation
2020-02-03 10:49:44 -05:00
Michael Schurter
5dbccce4de sentinel: copy jobs to prevent mutation
It's unclear whether Sentinel code can mutate values passed to the eval,
so ensure it cannot by copying the job.
2020-02-03 08:48:51 -05:00
Michael Lange
ce3d58191d Merge pull request #7047 from hashicorp/f-ui/node-drain-icons
UI: Node drain status light icons
2020-01-31 22:45:04 -08:00
Seth Hoenig
f5eba8d3b8 Merge pull request #7027 from hashicorp/dev-connect-acls
connect acls - rebase all the things
2020-01-31 19:37:06 -06:00
Seth Hoenig
37dc755af9 nomad/docs: increment version number to 0.10.4 2020-01-31 19:06:46 -06:00
Seth Hoenig
a3476f7416 docs: update chanagelog to mention connect with acls 2020-01-31 19:06:42 -06:00
Seth Hoenig
1c15ad9ed8 nomad: remove unused default schedular variable
This is from a merge conflict resolution that went the wrong direction.

I assumed the block had been added, but really it had been removed. Now,
it is removed once again.
2020-01-31 19:06:37 -06:00
Seth Hoenig
1fa551838f nomad: min cluster version for connect ACLs is now v0.10.4 2020-01-31 19:06:19 -06:00
Seth Hoenig
1f8e31770c tests: set consul token for nomad client for testing SIDS TR hook 2020-01-31 19:06:15 -06:00
Seth Hoenig
04b526662c e2e: setup consul ACLs a little more correctly 2020-01-31 19:06:11 -06:00
Seth Hoenig
b0e3acac37 e2e: remove redundant extra API call for getting allocs 2020-01-31 19:06:07 -06:00
Seth Hoenig
9fa02763ac e2e: agent token was only being set for server0 2020-01-31 19:06:03 -06:00
Seth Hoenig
8372bfbc8b e2e: use hclfmt on consul acls policy config files 2020-01-31 19:05:59 -06:00
Seth Hoenig
390a7f1c24 e2e: uncomment test case that is not broken 2020-01-31 19:05:55 -06:00
Seth Hoenig
d252bb4e80 e2e: do not use eventually when waiting for allocs
This test is causing panics. Unlike the other similar tests, this
one is using require.Eventually which is doing something bad, and
this change replaces it with a for-loop like the other tests.

Failure:

=== RUN   TestE2E/Connect
=== RUN   TestE2E/Connect/*connect.ConnectE2ETest
=== RUN   TestE2E/Connect/*connect.ConnectE2ETest/TestConnectDemo
=== RUN   TestE2E/Connect/*connect.ConnectE2ETest/TestMultiServiceConnect
=== RUN   TestE2E/Connect/*connect.ConnectClientStateE2ETest
panic: Fail in goroutine after TestE2E/Connect/*connect.ConnectE2ETest has completed

goroutine 38 [running]:
testing.(*common).Fail(0xc000656500)
	/opt/google/go/src/testing/testing.go:565 +0x11e
testing.(*common).Fail(0xc000656100)
	/opt/google/go/src/testing/testing.go:559 +0x96
testing.(*common).FailNow(0xc000656100)
	/opt/google/go/src/testing/testing.go:587 +0x2b
testing.(*common).Fatalf(0xc000656100, 0x1512f90, 0x10, 0xc000675f88, 0x1, 0x1)
	/opt/google/go/src/testing/testing.go:672 +0x91
github.com/hashicorp/nomad/e2e/connect.(*ConnectE2ETest).TestMultiServiceConnect.func1(0x0)
	/home/shoenig/go/src/github.com/hashicorp/nomad/e2e/connect/multi_service.go:72 +0x296
github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert.Eventually.func1(0xc0004962a0, 0xc0002338f0)
	/home/shoenig/go/src/github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert/assertions.go:1494 +0x27
created by github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert.Eventually
	/home/shoenig/go/src/github.com/hashicorp/nomad/vendor/github.com/stretchr/testify/assert/assertions.go:1493 +0x272
FAIL	github.com/hashicorp/nomad/e2e	21.427s
2020-01-31 19:05:47 -06:00
Seth Hoenig
1c9500ab27 e2e: remove forgotten unused field from new struct 2020-01-31 19:05:41 -06:00
Seth Hoenig
6d99dd4841 e2e: e2e test for connect with consul acls
Provide script for managing Consul ACLs on a TF provisioned cluster for
e2e testing. Script can be used to 'enable' or 'disable' Consul ACLs,
and automatically takes care of the bootstrapping process if necessary.

The bootstrapping process takes a long time, so we may need to
extend the overall e2e timeout (20 minutes seems fine).

Introduces basic tests for Consul Connect with ACLs.
2020-01-31 19:05:36 -06:00
Seth Hoenig
0f285b840e tests: skip some SIDS hook tests if running tests as root 2020-01-31 19:05:32 -06:00
Seth Hoenig
08951ac759 client: additional test cases around failures in SIDS hook 2020-01-31 19:05:27 -06:00
Seth Hoenig
91c7dbaa8d client: PR cleanup - improved logging around kill task in SIDS hook 2020-01-31 19:05:23 -06:00
Seth Hoenig
f8949dde35 client: PR cleanup - shadow context variable 2020-01-31 19:05:19 -06:00
Seth Hoenig
0589b656b7 nomad: make TaskGroup.UsesConnect helper a public helper 2020-01-31 19:05:11 -06:00
Seth Hoenig
6152ded0f4 nomad: fix leftover missed refactoring in consul policy checking 2020-01-31 19:05:06 -06:00
Seth Hoenig
40de85867d client: manage TR kill from parent on SI token derivation failure
Re-orient the management of the tr.kill to happen in the parent of
the spawned goroutine that is doing the actual token derivation. This
makes the code a little more straightforward, making it easier to
reason about not leaking the worker goroutine.
2020-01-31 19:05:02 -06:00
Seth Hoenig
1fca495a85 client: set context timeout around SI token derivation
The derivation of an SI token needs to be safegaurded by a context
timeout, otherwise an unresponsive Consul could cause the siHook
to block forever on Prestart.
2020-01-31 19:04:56 -06:00
Seth Hoenig
bbedeb670d nomad,client: apply more comment/style PR tweaks 2020-01-31 19:04:52 -06:00
Seth Hoenig
cc7b768907 nomad,client: apply smaller PR suggestions
Apply smaller suggestions like doc strings, variable names, etc.

Co-Authored-By: Nick Ethier <nethier@hashicorp.com>
Co-Authored-By: Michael Schurter <mschurter@hashicorp.com>
2020-01-31 19:04:40 -06:00
Seth Hoenig
d24d470775 comments: cleanup some leftover debug comments and such 2020-01-31 19:04:35 -06:00
Seth Hoenig
e825a0f769 client: skip task SI token file load failure if testing as root
The TestEnvoyBootstrapHook_maybeLoadSIToken test case only works when
running as a non-priveleged user, since it deliberately tries to read
an un-readable file to simulate a failure loading the SI token file.
2020-01-31 19:04:30 -06:00
Seth Hoenig
4b4dfacda5 client: remove unused indirection for referencing consul executable
Was thinking about using the testing pattern where you create executable
shell scripts as test resources which "mock" the process a bit of code
is meant to fork+exec. Turns out that wasn't really necessary in this case.
2020-01-31 19:04:25 -06:00
Seth Hoenig
ead935d12c agent: re-enable the server in dev mode 2020-01-31 19:04:19 -06:00
Seth Hoenig
9f48d83378 nomad: handle SI token revocations concurrently
Be able to revoke SI token accessors concurrently, and also
ratelimit the requests being made to Consul for the various
ACL API uses.
2020-01-31 19:04:14 -06:00
Seth Hoenig
d85cccc8d0 nomad: fixup token policy validation 2020-01-31 19:04:08 -06:00
Seth Hoenig
6bc6a52f99 client: enable envoy bootstrap hook to set SI token
When creating the envoy bootstrap configuration, we should append
the "-token=<token>" argument in the case where the sidsHook placed
the token in the secrets directory.
2020-01-31 19:04:01 -06:00
Seth Hoenig
674ccaa122 nomad: proxy requests for Service Identity tokens between Clients and Consul
Nomad jobs may be configured with a TaskGroup which contains a Service
definition that is Consul Connect enabled. These service definitions end
up establishing a Consul Connect Proxy Task (e.g. envoy, by default). In
the case where Consul ACLs are enabled, a Service Identity token is required
for these tasks to run & connect, etc. This changeset enables the Nomad Server
to recieve RPC requests for the derivation of SI tokens on behalf of instances
of Consul Connect using Tasks. Those tokens are then relayed back to the
requesting Client, which then injects the tokens in the secrets directory of
the Task.
2020-01-31 19:03:53 -06:00
Seth Hoenig
f8666bb1f9 client: enable nomad client to request and set SI tokens for tasks
When a job is configured with Consul Connect aware tasks (i.e. sidecar),
the Nomad Client should be able to request from Consul (through Nomad Server)
Service Identity tokens specific to those tasks.
2020-01-31 19:03:38 -06:00
Seth Hoenig
560ce96b90 nomad: ensure a unique ClusterID exists when leader (gh-6702)
Enable any Server to lookup the unique ClusterID. If one has not been
generated, and this node is the leader, generate a UUID and attempt to
apply it through raft.

The value is not yet used anywhere in this changeset, but is a prerequisite
for gh-6701.
2020-01-31 19:03:26 -06:00
Seth Hoenig
0040c75e8e command, docs: create and document consul token configuration for connect acls (gh-6716)
This change provides an initial pass at setting up the configuration necessary to
enable use of Connect with Consul ACLs. Operators will be able to pass in a Consul
Token through `-consul-token` or `$CONSUL_TOKEN` in the `job run` and `job revert`
commands (similar to Vault tokens).

These values are not actually used yet in this changeset.
2020-01-31 19:02:53 -06:00
Michael Lange
4419510652 Call out the 'down' status too, since it's a pretty bad one 2020-01-31 12:56:15 -08:00
Michael Lange
2e2bcc50a7 Add an animation for the initializing state 2020-01-31 12:56:11 -08:00
Michael Lange
e95370464a Redo the node-status-light CSS to be icon-based 2020-01-31 12:56:08 -08:00
Michael Lange
7f60198742 New node initializing icon 2020-01-31 12:56:05 -08:00
Michael Lange
ec01c0bdd7 Assign icons to node statuses 2020-01-31 12:56:02 -08:00
Michael Lange
3d5193b8de Add an icon inside the node status light 2020-01-31 12:55:59 -08:00
Michael Lange
2f1dec2924 Allow for an icon within the node status light 2020-01-31 12:55:55 -08:00