Commit Graph

3684 Commits

Author SHA1 Message Date
Mahmood Ali
c0162fab35 move cstructs.DeviceNetwork to drivers pkg 2019-01-08 09:11:47 -05:00
Mahmood Ali
694e3010c2 use drivers.FSIsolation 2019-01-08 09:11:47 -05:00
Mahmood Ali
2831088fc5 remove deprecated allocrunner 2019-01-08 09:11:47 -05:00
Mahmood Ali
607e7f2dde remove always false parameter
Simplify allocDir.Build() function to avoid depending on client/structs,
and remove a parameter that's always set to `false`.

The motivation here is to avoid a dependency cycle between
drivers/cstructs and alloc_dir.
2019-01-08 09:11:47 -05:00
Danielle Tomlinson
994057abf4 drivers: Add internal interface for Shutdown
This allows us to correctly terminate internal state during runs of the
nomad test suite, e.g closing eventer contexts correctly.
2019-01-08 13:48:49 +01:00
Alex Dadgar
428cec9682 Merge pull request #5152 from hashicorp/f-recover
Task runner recovers from external plugin exiting
2019-01-07 15:27:33 -08:00
Alex Dadgar
6bb99c93d0 Review comments 2019-01-07 14:50:28 -08:00
Alex Dadgar
5424d3b540 vet 2019-01-07 14:49:41 -08:00
Alex Dadgar
19e67a0916 Test recovery 2019-01-07 14:49:41 -08:00
Alex Dadgar
144866a87b Mock driver has recovery, stats 2019-01-07 14:49:40 -08:00
Alex Dadgar
b300306c4a comments 2019-01-07 14:49:40 -08:00
Alex Dadgar
3257eb6d86 Fix hooks 2019-01-07 14:49:40 -08:00
Alex Dadgar
437f03d877 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar
bdf4e1f89a Don't log event error on driver shutdown 2019-01-07 14:49:40 -08:00
Michael Schurter
3e7fa0072a Merge pull request #5043 from hashicorp/b-taskenv-conflicts
taskenv: have maps take precedence over primitives
2019-01-07 12:34:48 -08:00
Mahmood Ali
90f747d895 tests: helper function for checking docker presense 2019-01-07 08:27:06 -05:00
Mahmood Ali
17b7490891 taskrunner: emit TaskReceived event
Preserve pre-0.9, where task runner emits `Received: Task received by
client` event on task runner creation.
2019-01-04 14:32:29 -05:00
Michael Schurter
d2c81c375c Merge pull request #5038 from hashicorp/b-drivermanager-tests
WIP: fix failing tests caused by async driver manager
2019-01-03 12:32:18 -08:00
Danielle Tomlinson
1209d03a5c Merge pull request #5142 from hashicorp/dani/cleanup-allocrunner-logs
allocrunner: Standardised discard logs
2019-01-03 18:40:48 +01:00
Preetha
5673a2c33d Merge pull request #5140 from hashicorp/dani/b-taskrunner
taskrunner: Persist environment from hooks
2019-01-03 09:30:52 -06:00
Danielle Tomlinson
acf2c524f3 allocrunner: Standardised discard logs
Follow up from https://github.com/hashicorp/nomad/pull/5007#pullrequestreview-186739124
2019-01-03 14:04:31 +01:00
Danielle Tomlinson
5dadb6cc7e chore: Fix environement->environment typo 2019-01-03 13:31:30 +01:00
Danielle Tomlinson
54dde24bcb taskrunner: Persist environment from hooks
https://github.com/hashicorp/nomad/pull/5032 introduced a regression
where the origHookState was used in place of the response from the hook.
2019-01-03 13:13:57 +01:00
Alex Dadgar
c54aad6231 Merge pull request #5032 from hashicorp/f-driver-env
Store device envs separately and pass to drivers
2018-12-20 13:38:27 -08:00
Michael Schurter
748bc58cb2 taskenv: have maps take precedence over primitives
**The Bug:**

You may have seen log lines like this when running 0.9.0-dev:

```
... client.alloc_runner.task_runner: some environment variables not available for rendering: ... keys="attr.driver.docker.volumes.enabled, attr.driver.docker.version, attr.driver.docker.bridge_ip, attr.driver.qemu.version"
```

Not only should we not be erroring on builtin driver attributes, but the
results were nondeterministic due to map iteration order!

The root cause is that we have an old root attribute for all drivers
like:

```
attr.driver.docker = "1"
```

When attributes were opaque variable names it was fine to also have
"nested" attributes like:

```
attr.driver.docker.version = "1.2.3"
```

However in the HCLv2 world the variable names are no longer opaque: they
form an object tree. The `docker` object can no longer both hold a value
(`"1"`) *and* nested attributes (`version = "1.2.3"`).

**The Fix:**

Since the old `attr.driver.<name> = "1"` attribues are useless for task
config interpolation, create a new precedence rule for creating the task
config evaluation context:

*Maps take precedence over primitives.*

This means `attr.driver.docker.version` will always take precedence over
`attr.driver.docker`. The results are determinstic and give users access
to the more useful metadata.

I made this a general precedence rule instead of special-casing driver
attrs because it seemed like better default behavior than spamming
WARNings to logs that were likely unactionable by users.
2018-12-20 11:37:46 -08:00
Nick Ethier
145827d8b7 fix tests that fail as a result of async client startup 2018-12-20 00:53:44 -05:00
Nick Ethier
848554ba61 client: add proper build flag to allocrunner testing.go 2018-12-19 20:22:07 -05:00
Michael Schurter
f2e8375e35 test: re-eanble periodic fingerprint test 2018-12-19 17:08:24 -08:00
Michael Schurter
2cf3a3c56a test: copy AR's Alloc before mutating
Fixes a race in client tests
2018-12-19 15:48:02 -08:00
Michael Schurter
6478a93aca drivermgr: fix race in building driver list 2018-12-19 15:48:02 -08:00
Michael Schurter
60522ac457 Merge pull request #5030 from hashicorp/test-client-statusupdate
client: assert alloc status updates work
2018-12-19 14:55:34 -08:00
Alex Dadgar
99df4c98c7 Store device envs separately and pass to drivers 2018-12-19 14:23:09 -08:00
Michael Schurter
d5f4de38f8 client: assert alloc status updates work
Re-enabling and updating an old test. Able to cut out a ton of extra
work by using WaitForRunning which does almost everything this test
needs.
2018-12-19 11:41:53 -08:00
Michael Schurter
f07fe3459a client/state: missing deploy status isn't an error
Fixes TestClient_SaveRestoreState
2018-12-19 10:39:27 -08:00
Michael Schurter
75a889244e tests: implement HasHealth for mock health 2018-12-19 10:39:27 -08:00
Michael Schurter
ab34578740 gofmt -s -w upgrade_int_test.go 2018-12-19 10:39:27 -08:00
Michael Schurter
fc7be6b7f0 client/state: improve upgradeTaskBucket error handling
And add a test
2018-12-19 10:39:27 -08:00
Michael Schurter
11e923e508 client/state: add context to errors
Unfortunately I don't know how to test these errors. As far as I can
tell they should only happen if there was a programming error in the
upgrade code or the underlying boltdb was corrupted somehow.
2018-12-19 10:39:27 -08:00
Michael Schurter
cba472485b client/state: use 2 as version; test error path 2018-12-19 10:39:27 -08:00
Michael Schurter
784706a1e5 client/state: support upgrading from 0.8->0.9
Also persist and load DeploymentStatus to avoid rechecking health after
client restarts.
2018-12-19 10:39:27 -08:00
Michael Schurter
bd53ccf548 client/state: reorg state buckets to ease transition
* Prefix task bucket with task- to prevent name conflicts
* Shorten device manager bucket name
* Remove commented out outdated var
* Update layout comment
2018-12-19 10:22:28 -08:00
Michael Schurter
3a41fd7b31 tr: fix HookState Copy() and Equal() methods
They did not take into account the Env field.
2018-12-19 09:58:06 -08:00
Danielle Tomlinson
b63095db60 allocrunner: Close updates routine correctly 2018-12-19 18:32:51 +01:00
Nick Ethier
c98aaad342 devicemanager: fix devicemanager tests 2018-12-19 00:35:12 -05:00
Nick Ethier
1ff3b681da drivermanager: use correct plugin config types 2018-12-18 23:07:01 -05:00
Nick Ethier
12528cadda drivermanager: attempt to reattach and shutdown driver plugin if blocked by allow/block lists 2018-12-18 23:01:57 -05:00
Nick Ethier
6951ca487d drivermanager: use allocID and task name to route task events 2018-12-18 23:01:51 -05:00
Nick Ethier
3100f7bd64 client/pluginmanager: add plugin manager interface to device/driver managers 2018-12-18 22:56:23 -05:00
Nick Ethier
331793e283 client: batch initial fingerprinting in plugin manangers
drivermanager: fix pr comments/feedback
2018-12-18 22:56:19 -05:00
Nick Ethier
2f010a2f25 client/drivermananger: fixup issues from rebase and address PR comments 2018-12-18 22:55:38 -05:00