Commit Graph

84 Commits

Author SHA1 Message Date
Mahmood Ali
4ca6cda6c1 clarify unknown signal log line (#5466) 2019-03-25 17:19:43 -04:00
Mahmood Ali
55ee03a2ad Merge pull request #5341 from hashicorp/ci-windows-docker
Run Docker tests in Windows AppVeyor CI
2019-02-21 13:17:33 -05:00
Mahmood Ali
d6250ec0d6 tests: IsTravis() -> IsCI()
Replace IsTravis() references that is intended for more CI environments
rather than for Travis environment specifically.
2019-02-20 08:21:03 -05:00
Mahmood Ali
afb0c5ac45 drivers: restore port_map old json support
This ensures that `port_map` along with other block like attribute
declarations (e.g. ulimit, labels, etc) can handle various hcl and json
syntax that was supported in 0.8.

In 0.8.7, the following declarations are effectively equivalent:

```
// hcl block
port_map {
  http = 80
  https = 443
}

// hcl assignment
port_map = {
  http  = 80
  https = 443
}

// json single element array of map (default in API response)
{"port_map": [{"http": 80, "https": 443}]}

// json array of individual maps (supported accidentally iiuc)
{"port_map: [{"http": 80}, {"https": 443}]}
```

We achieve compatbility by using `NewAttr("...", "list(map(string))",
false)` to be serialized to a `map[string]string` wrapper, instead of using
`BlockAttrs` declaration.  The wrapper merges the list of maps
automatically, to ease driver development.

This approach is closer to how v0.8.7 implemented the fields [1][2], and
despite its verbosity, seems to perserve 0.8.7 behavior in hcl2.

This is only required for built-in types that have backward
compatibility constraints.  External drivers should use `BlockAttrs`
instead, as they see fit.

[1] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L216
[2] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L698-L700
2019-02-16 11:37:33 -05:00
Mahmood Ali
941f89e0fe tests: add hcl task driver config parsing tests (#5314)
* drivers: add config parsing tests

Add basic tests for parsing and encoding task config.

* drivers/docker: fix some config declarations

* refactor and document config parse helpers
2019-02-12 14:46:37 -05:00
Nick Ethier
aa713a4d9f client: fix bug during 0.8 state up grade that causes external drivers to fail 2019-01-30 14:22:29 -05:00
Nick Ethier
bed9efae44 Merge branch 'master' into f-driver-upgradepath-test
* master: (23 commits)
  tests: avoid assertion in goroutine
  spell check
  ci: run checkscripts
  tests: deflake TestRktDriver_StartWaitRecoverWaitStop
  drivers/rkt: Remove unused github.com/rkt/rkt
  drivers/rkt: allow development on non-linux
  cli: Hide `nomad docker_logger` from help output
  api: test api and structs are in sync
  goimports until make check is happy
  nil check node resources to prevent panic
  tr: use context in as select statement
  move pluginutils -> helper/pluginutils
  vet
  goimports
  gofmt
  Split hclspec
  move hclutils
  Driver tests do not use hcl2/hcl, hclspec, or hclutils
  move reattach config
  loader and singleton
  ...
2019-01-23 21:01:24 -05:00
Nick Ethier
a9060f44eb drivers: add docker upgrade path and e2e test 2019-01-23 14:44:42 -05:00
Mahmood Ali
4781f1dd7b Merge pull request #5229 from hashicorp/r-grabbag-201901019
Grab bag of small changes
2019-01-23 13:06:51 -05:00
Mahmood Ali
4817a89bd3 tests: avoid assertion in goroutine 2019-01-23 12:34:16 -05:00
Mahmood Ali
9c94953c08 tests: deflake TestRktDriver_StartWaitRecoverWaitStop 2019-01-23 10:54:49 -05:00
Mahmood Ali
597fb32dde drivers/rkt: allow development on non-linux
Avoid depending on OS-specific library for some simple structs.
2019-01-23 10:53:49 -05:00
Michael Schurter
158c74887e goimports until make check is happy 2019-01-23 06:27:14 -08:00
Michael Schurter
0d61ff0fb9 move pluginutils -> helper/pluginutils
I wanted a different color bikeshed, so I get to paint it
2019-01-22 15:50:08 -08:00
Alex Dadgar
e46d67a889 Driver tests do not use hcl2/hcl, hclspec, or hclutils 2019-01-22 15:43:34 -08:00
Alex Dadgar
2d23f4a038 move reattach config 2019-01-22 15:11:58 -08:00
Alex Dadgar
c19cd2e5cf loader and singleton 2019-01-22 15:11:57 -08:00
Nick Ethier
82d16f8f6c drivers: fix func naming 2019-01-18 18:31:02 -05:00
Nick Ethier
994c66f7d7 drivers: use consts for task handle version 2019-01-18 18:31:01 -05:00
Nick Ethier
1d50ced70a drivers: add upgrade path for rawexec, java, rkt and qemu 2019-01-18 18:31:01 -05:00
Nick Ethier
07cdedec2f driver: add pre09 migration logic 2019-01-18 18:31:01 -05:00
Michael Schurter
10068422e0 Merge pull request #5197 from hashicorp/b-rkt-cpu
rkt: revert to pre-0.9 --cpu flag
2019-01-16 15:05:21 -08:00
Preetha Appan
3e16b52361 clean up read access 2019-01-16 11:04:11 -06:00
Preetha Appan
80c00fc268 Refactor logging in drivers to use a tri-state boolean
Changes logging warnings/errors only if the state changes
from healthy to unhealthy
2019-01-16 10:19:31 -06:00
Michael Schurter
97be19b176 rkt: revert to pre-0.9 --cpu flag
See
https://github.com/hashicorp/nomad/issues/3394#issuecomment-453296121
for details. During 0.9 development we switched to shares, but we'd
prefer to maintain backward compat.
2019-01-15 13:15:28 -08:00
Alex Dadgar
109c5ef650 Merge pull request #5173 from hashicorp/b-log-levels
Plugins use parent loggers
2019-01-14 16:14:30 -08:00
Nick Ethier
fbf9a4c772 executor: implement streaming stats API
plugins/driver: update driver interface to support streaming stats

client/tr: use streaming stats api

TODO:
 * how to handle errors and closed channel during stats streaming
 * prevent tight loop if Stats(ctx) returns an error

drivers: update drivers TaskStats RPC to handle streaming results

executor: better error handling in stats rpc

docker: better control and error handling of stats rpc

driver: allow stats to return a recoverable error
2019-01-12 12:18:22 -05:00
Alex Dadgar
e3859dc5e5 fix rkt use of executor 2019-01-11 11:36:37 -08:00
Mahmood Ali
d1fbd735f3 Merge pull request #5157 from hashicorp/r-drivers-no-cstructs
drivers: avoid referencing client/structs package
2019-01-09 13:06:46 -05:00
Mahmood Ali
7840ab51e2 fix more cases of logging arity errors 2019-01-09 09:22:47 -05:00
Mahmood Ali
800a3522e3 drivers: re-export ResourceUsage structs
Re-export the ResourceUsage structs in drivers package to avoid drivers
directly depending on the internal client/structs package directly.

I attempted moving the structs to drivers, but that caused some import
cycles that was a bit hard to disentagle.  Alternatively, I added an
alias here that's sufficient for our purposes of avoiding external
drivers depend on internal packages, while allowing us to restructure
packages in future without breaking source compatibility.
2019-01-08 09:11:47 -05:00
Mahmood Ali
c0162fab35 move cstructs.DeviceNetwork to drivers pkg 2019-01-08 09:11:47 -05:00
Mahmood Ali
694e3010c2 use drivers.FSIsolation 2019-01-08 09:11:47 -05:00
Danielle Tomlinson
476e44b4e4 drivers: Implement InternalPluginDriver interface
This implements the InternalPluginDriver interface in each driver, and
calls the cancellation fn for their respective eventers.

This fixes a per task goroutine leak during test suite execution.
2019-01-08 13:49:31 +01:00
Alex Dadgar
6bb99c93d0 Review comments 2019-01-07 14:50:28 -08:00
Alex Dadgar
437f03d877 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar
06a52b57e6 rkt fingerprint logs once 2019-01-07 14:49:40 -08:00
Preetha Appan
26594aa31e Standardize driver health description messages for all drivers 2019-01-06 22:06:38 -06:00
Preetha Appan
b8cbcdb371 remove unnecessary logging in rkt driver fingerprint method 2019-01-06 20:59:20 -06:00
Danielle Tomlinson
aa6b170506 rkt: Return consistent error when not root 2018-12-20 13:02:46 +01:00
Alex Dadgar
07a7555acd lint 2018-12-18 16:48:00 -08:00
Alex Dadgar
ed4f8eac6e Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Alex Dadgar
cd6879409c Drivers 2018-12-18 15:50:11 -08:00
Mahmood Ali
ee652be312 tests: fix rkt command environment (#5011)
The environment variables needed for envoking `rkt` command line
should include host PATH (to access `iptables`).

Given that the command runs outside the VM, untrusted task environment
variables should NOT be honored here.

We do this already with `rkt`, but the change is quite subtle to miss
when refactoring.
2018-12-15 20:25:36 -05:00
Nick Ethier
8a344412e8 Merge branch 'master' into f-grpc-executor
* master: (71 commits)
  Fix output of 'nomad deployment fail' with no arg
  Always create a running allocation when testing task state
  tests: ensure exec tests pass valid task resources (#4992)
  some changes for more idiomatic code
  fix iops related tests
  fixed bug in loop delay
  gofmt
  improved code for readability
  client: updateAlloc release lock after read
  fixup! device attributes in `nomad node status -verbose`
  drivers/exec: support device binds and mounts
  fix iops bug and increase test matrix coverage
  tests: tag image explicitly
  changelog
  ci: install lxc-templates explicitly
  tests: skip checking rdma cgroup
  ci: use Ubuntu 16.04 (Xenial) in TravisCI
  client: update driver info on new fingerprint
  drivers/docker: enforce volumes.enabled (#4983)
  client: Style: use fluent style for building loggers
  ...
2018-12-13 14:41:09 -05:00
Mahmood Ali
d31e52b1f6 tests: update stop/kill tests with new pattern
Update rawexec and rkt stop/kill tests with the patterns introduced in
7a49e9b68e.  This implementation should be
more resilient to discrepancy between task stopping and task being marked as exited.
2018-12-11 09:35:26 -05:00
Nick Ethier
6b39bb33b6 Merge branch 'master' into f-grpc-executor 2018-12-06 21:42:38 -05:00
Nick Ethier
9e3c2492e5 executor: fix tests 2018-12-06 21:39:53 -05:00
Nick Ethier
13b582fcfb executor: merge plugin shim with executor package 2018-12-06 21:13:45 -05:00
Nick Ethier
0087a51a7a executor: remove structs package 2018-12-06 20:54:14 -05:00