Commit Graph

3498 Commits

Author SHA1 Message Date
Alex Dadgar
c7fc39d38d Merge pull request #5168 from hashicorp/b-kill-race
Improve Kill handling on task runner
2019-01-09 12:05:10 -08:00
Alex Dadgar
d916c0dd12 add more comments 2019-01-09 12:04:22 -08:00
Michael Schurter
e44d51f4d0 Spelling fix
Co-Authored-By: dadgar <alex@hashicorp.com>
2019-01-09 11:42:40 -08:00
Mahmood Ali
d1fbd735f3 Merge pull request #5157 from hashicorp/r-drivers-no-cstructs
drivers: avoid referencing client/structs package
2019-01-09 13:06:46 -05:00
Mahmood Ali
2349a01274 Merge pull request #5163 from hashicorp/r-minor-changes-20180108
Fix a panic on node.Deregister fail
2019-01-09 09:56:00 -05:00
Mahmood Ali
7840ab51e2 fix more cases of logging arity errors 2019-01-09 09:22:47 -05:00
Mahmood Ali
76d40947d7 Merge pull request #5159 from hashicorp/r-macos-tests
Fix Travis MacOS job
2019-01-09 08:22:30 -05:00
Alex Dadgar
a5ba15591a Improve Kill handling on task runner
This PR improves how killing a task is handled. Before the kill function
directly orchestrated the killing and was only valid while the task was
running. The new behavior is to mark the desired state and wait for the
task runner to converge to that state.
2019-01-08 16:42:26 -08:00
Mahmood Ali
8a96bf84a1 tests: fix a test job constaints failing in macOS
Allow scheduling mock job when running on MacOS (or Windows) hosts.
2019-01-08 12:37:42 -05:00
Michael Schurter
6c561efdf8 Merge pull request #5151 from hashicorp/b-task-events
Emit Killing task events and add e2e tests
2019-01-08 09:33:04 -08:00
Mahmood Ali
7b3ff40d22 run gofmt 2019-01-08 11:15:38 -05:00
Michael Schurter
ef2c835434 Merge pull request #5035 from hashicorp/test-client
test: re-eanble periodic fingerprint test
2019-01-08 07:37:39 -08:00
Michael Schurter
1ae8261139 client: emit Killing/Killed task events
We were just emitting Killed/Terminated events before. In v0.8 we
emitted Killing/Killed, but lacked Terminated when explicitly stopping
a task. This change makes it so Terminated is always included, whether
explicitly stopping a task or it exiting on its own.

New output:

2019-01-04T14:58:51-08:00  Killed            Task successfully killed
2019-01-04T14:58:51-08:00  Terminated        Exit Code: 130, Signal: 2
2019-01-04T14:58:51-08:00  Killing           Sent interrupt
2019-01-04T14:58:51-08:00  Leader Task Dead  Leader Task in Group dead
2019-01-04T14:58:49-08:00  Started           Task started by client
2019-01-04T14:58:49-08:00  Task Setup        Building Task Directory
2019-01-04T14:58:49-08:00  Received          Task received by client

Old (v0.8.6) output:

2019-01-04T22:14:54Z  Killed            Task successfully killed
2019-01-04T22:14:54Z  Killing           Sent interrupt. Waiting 5s before force killing
2019-01-04T22:14:54Z  Leader Task Dead  Leader Task in Group dead
2019-01-04T22:14:53Z  Started           Task started by client
2019-01-04T22:14:53Z  Task Setup        Building Task Directory
2019-01-04T22:14:53Z  Received          Task received by client
2019-01-08 07:20:54 -08:00
Michael Schurter
838c8fcc75 Merge pull request #5034 from hashicorp/test-fix-races
Test fix races
2019-01-08 07:04:09 -08:00
Mahmood Ali
c0162fab35 move cstructs.DeviceNetwork to drivers pkg 2019-01-08 09:11:47 -05:00
Mahmood Ali
694e3010c2 use drivers.FSIsolation 2019-01-08 09:11:47 -05:00
Mahmood Ali
2831088fc5 remove deprecated allocrunner 2019-01-08 09:11:47 -05:00
Mahmood Ali
607e7f2dde remove always false parameter
Simplify allocDir.Build() function to avoid depending on client/structs,
and remove a parameter that's always set to `false`.

The motivation here is to avoid a dependency cycle between
drivers/cstructs and alloc_dir.
2019-01-08 09:11:47 -05:00
Danielle Tomlinson
994057abf4 drivers: Add internal interface for Shutdown
This allows us to correctly terminate internal state during runs of the
nomad test suite, e.g closing eventer contexts correctly.
2019-01-08 13:48:49 +01:00
Alex Dadgar
428cec9682 Merge pull request #5152 from hashicorp/f-recover
Task runner recovers from external plugin exiting
2019-01-07 15:27:33 -08:00
Alex Dadgar
6bb99c93d0 Review comments 2019-01-07 14:50:28 -08:00
Alex Dadgar
5424d3b540 vet 2019-01-07 14:49:41 -08:00
Alex Dadgar
19e67a0916 Test recovery 2019-01-07 14:49:41 -08:00
Alex Dadgar
144866a87b Mock driver has recovery, stats 2019-01-07 14:49:40 -08:00
Alex Dadgar
b300306c4a comments 2019-01-07 14:49:40 -08:00
Alex Dadgar
3257eb6d86 Fix hooks 2019-01-07 14:49:40 -08:00
Alex Dadgar
437f03d877 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar
bdf4e1f89a Don't log event error on driver shutdown 2019-01-07 14:49:40 -08:00
Michael Schurter
3e7fa0072a Merge pull request #5043 from hashicorp/b-taskenv-conflicts
taskenv: have maps take precedence over primitives
2019-01-07 12:34:48 -08:00
Mahmood Ali
90f747d895 tests: helper function for checking docker presense 2019-01-07 08:27:06 -05:00
Mahmood Ali
17b7490891 taskrunner: emit TaskReceived event
Preserve pre-0.9, where task runner emits `Received: Task received by
client` event on task runner creation.
2019-01-04 14:32:29 -05:00
Michael Schurter
d2c81c375c Merge pull request #5038 from hashicorp/b-drivermanager-tests
WIP: fix failing tests caused by async driver manager
2019-01-03 12:32:18 -08:00
Danielle Tomlinson
1209d03a5c Merge pull request #5142 from hashicorp/dani/cleanup-allocrunner-logs
allocrunner: Standardised discard logs
2019-01-03 18:40:48 +01:00
Preetha
5673a2c33d Merge pull request #5140 from hashicorp/dani/b-taskrunner
taskrunner: Persist environment from hooks
2019-01-03 09:30:52 -06:00
Danielle Tomlinson
acf2c524f3 allocrunner: Standardised discard logs
Follow up from https://github.com/hashicorp/nomad/pull/5007#pullrequestreview-186739124
2019-01-03 14:04:31 +01:00
Danielle Tomlinson
5dadb6cc7e chore: Fix environement->environment typo 2019-01-03 13:31:30 +01:00
Danielle Tomlinson
54dde24bcb taskrunner: Persist environment from hooks
https://github.com/hashicorp/nomad/pull/5032 introduced a regression
where the origHookState was used in place of the response from the hook.
2019-01-03 13:13:57 +01:00
Alex Dadgar
c54aad6231 Merge pull request #5032 from hashicorp/f-driver-env
Store device envs separately and pass to drivers
2018-12-20 13:38:27 -08:00
Michael Schurter
748bc58cb2 taskenv: have maps take precedence over primitives
**The Bug:**

You may have seen log lines like this when running 0.9.0-dev:

```
... client.alloc_runner.task_runner: some environment variables not available for rendering: ... keys="attr.driver.docker.volumes.enabled, attr.driver.docker.version, attr.driver.docker.bridge_ip, attr.driver.qemu.version"
```

Not only should we not be erroring on builtin driver attributes, but the
results were nondeterministic due to map iteration order!

The root cause is that we have an old root attribute for all drivers
like:

```
attr.driver.docker = "1"
```

When attributes were opaque variable names it was fine to also have
"nested" attributes like:

```
attr.driver.docker.version = "1.2.3"
```

However in the HCLv2 world the variable names are no longer opaque: they
form an object tree. The `docker` object can no longer both hold a value
(`"1"`) *and* nested attributes (`version = "1.2.3"`).

**The Fix:**

Since the old `attr.driver.<name> = "1"` attribues are useless for task
config interpolation, create a new precedence rule for creating the task
config evaluation context:

*Maps take precedence over primitives.*

This means `attr.driver.docker.version` will always take precedence over
`attr.driver.docker`. The results are determinstic and give users access
to the more useful metadata.

I made this a general precedence rule instead of special-casing driver
attrs because it seemed like better default behavior than spamming
WARNings to logs that were likely unactionable by users.
2018-12-20 11:37:46 -08:00
Nick Ethier
145827d8b7 fix tests that fail as a result of async client startup 2018-12-20 00:53:44 -05:00
Nick Ethier
848554ba61 client: add proper build flag to allocrunner testing.go 2018-12-19 20:22:07 -05:00
Michael Schurter
f2e8375e35 test: re-eanble periodic fingerprint test 2018-12-19 17:08:24 -08:00
Michael Schurter
2cf3a3c56a test: copy AR's Alloc before mutating
Fixes a race in client tests
2018-12-19 15:48:02 -08:00
Michael Schurter
6478a93aca drivermgr: fix race in building driver list 2018-12-19 15:48:02 -08:00
Michael Schurter
60522ac457 Merge pull request #5030 from hashicorp/test-client-statusupdate
client: assert alloc status updates work
2018-12-19 14:55:34 -08:00
Alex Dadgar
99df4c98c7 Store device envs separately and pass to drivers 2018-12-19 14:23:09 -08:00
Michael Schurter
d5f4de38f8 client: assert alloc status updates work
Re-enabling and updating an old test. Able to cut out a ton of extra
work by using WaitForRunning which does almost everything this test
needs.
2018-12-19 11:41:53 -08:00
Michael Schurter
f07fe3459a client/state: missing deploy status isn't an error
Fixes TestClient_SaveRestoreState
2018-12-19 10:39:27 -08:00
Michael Schurter
75a889244e tests: implement HasHealth for mock health 2018-12-19 10:39:27 -08:00
Michael Schurter
ab34578740 gofmt -s -w upgrade_int_test.go 2018-12-19 10:39:27 -08:00