Commit Graph

274 Commits

Author SHA1 Message Date
Danielle Tomlinson
5637d81642 docker: Fix missing import 2019-01-17 18:44:27 +01:00
Danielle Tomlinson
3287c3f019 chore: General Cleanup 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
8966c20155 docker: Only run Cleanup test on unix os' 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
2b3f5f2cb7 chore: goimports exec driver 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
070558e18d fixup: Typo in docker test 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
0bc1dbef56 chore: Fix docklog linting 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
3f3eb68a27 chore: Fix docker test linting
Due to https://github.com/tsenart/deadcode/issues/3 we can't specify
these consts on their own. This moves them into the _platform_test.go
files to avoid creating a package that only exposes a couple of values.
2019-01-17 18:43:14 +01:00
Danielle Tomlinson
8731fe3730 drivers/exec: SIGINT unavailable on windows 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
0ecef70715 rawexec: Fix Exec test on windows 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
ffcd76f03f rawexec: SIGINT is not available on Windows 2019-01-17 18:43:14 +01:00
Danielle Tomlinson
1e0825388e docker: Test cleanup for windows
* Docker for Windows does not support ulimits
* Use filepath.ToSlash to test workdir
* Convert expected mount paths to system style
* Skip security-opt test on windows
  - Windows does not support seccomp, and it's unclear which options are
    available.
* Skip StartN due to lack of sigint
* docker: Use api to get image info on windows
* No bridge on windows
* Stop hardcoding /bin/
2019-01-17 18:43:14 +01:00
Danielle Tomlinson
76ee0240ab docker: ExpandPath tests validate slashpath 2019-01-17 18:43:13 +01:00
Danielle Tomlinson
a587be44a7 dockerlogger: Fix tests on windows
Uses the home directory and windows path expansion, as c:\tmp doesn't
necessarily exist, and mktemp would involve unnecessarily complicating
the commands.
2019-01-17 18:43:13 +01:00
Danielle Tomlinson
347b4ad247 Expand unix build definition 2019-01-17 18:43:13 +01:00
Mahmood Ali
3c9de20820 fix imports 2019-01-16 19:53:48 -05:00
Michael Schurter
10068422e0 Merge pull request #5197 from hashicorp/b-rkt-cpu
rkt: revert to pre-0.9 --cpu flag
2019-01-16 15:05:21 -08:00
Preetha Appan
3e16b52361 clean up read access 2019-01-16 11:04:11 -06:00
Preetha Appan
80c00fc268 Refactor logging in drivers to use a tri-state boolean
Changes logging warnings/errors only if the state changes
from healthy to unhealthy
2019-01-16 10:19:31 -06:00
Preetha Appan
35ab26658c Make docker driver logging less redundant 2019-01-16 10:16:57 -06:00
Michael Schurter
97be19b176 rkt: revert to pre-0.9 --cpu flag
See
https://github.com/hashicorp/nomad/issues/3394#issuecomment-453296121
for details. During 0.9 development we switched to shares, but we'd
prefer to maintain backward compat.
2019-01-15 13:15:28 -08:00
oleksii.shyman
cc98f282d4 Add support for docker runtimes
- docker fingerprint issues a docker api system info call to get the
  list of supported OCI runtimes.
  - OCI runtimes are reported as comma separated list of names
  - docker driver is aware of GPU runtime presence
  - docker driver throws an error when user tries to run container with
  GPU, when GPU runtime is not present
  - docker GPU runtime name is configurable
2019-01-15 11:34:47 -08:00
Danielle Tomlinson
9cf6e1ae27 Merge pull request #5192 from hashicorp/dani/executor-close
executor: Always close stdout/stderr fifos
2019-01-15 17:49:04 +01:00
Danielle Tomlinson
fb1e24d2b9 Merge pull request #5184 from hashicorp/dani/b-logmon-reattach
docker: Terminate dockerlogger
2019-01-15 16:48:40 +01:00
Danielle Tomlinson
17dbace46b executor: Always close stdout/stderr fifos 2019-01-15 16:47:27 +01:00
Danielle Tomlinson
f9a4594095 docker: Terminate dockerlogger
Previously, we did not attempt to stop Docker Logger processes until
DestroyTask, which means that under many circumstances, we will never
successfully close the plugin client.

This commit terminates the plugin process when `run` terminates, or when
`DestroyTask` is called.

Steps to repro:

```
$ nomad agent -dev
$ nomad init
$ nomad run example.nomad
$ nomad stop example
$ ps aux | grep nomad # See docker logger process running
$ signal the dev agent
$ ps aux | grep nomad # See docker logger process running
```
2019-01-15 14:58:05 +01:00
Mahmood Ali
2cc4466573 propogate logs to executor plugin 2019-01-15 08:25:03 -05:00
Alex Dadgar
109c5ef650 Merge pull request #5173 from hashicorp/b-log-levels
Plugins use parent loggers
2019-01-14 16:14:30 -08:00
Nick Ethier
97a73e0116 Merge pull request #5018 from hashicorp/f-executor-stats
executor: streaming stats api
2019-01-14 15:02:35 -05:00
Nick Ethier
d086bb2d20 qemu: missing gofmt 2019-01-13 16:06:56 -05:00
Michael Schurter
60acfc8289 Update drivers/qemu/driver_test.go
use t.Logf instead of fmt.Printf

Co-Authored-By: nickethier <ncethier@gmail.com>
2019-01-12 21:33:55 -05:00
Nick Ethier
72a4685534 drivers: plumb grpc client logger 2019-01-12 12:18:23 -05:00
Nick Ethier
9904463da2 executor: fix failing stats related test 2019-01-12 12:18:23 -05:00
Nick Ethier
f6af1d4d04 docker: add test for stats collection 2019-01-12 12:18:22 -05:00
Nick Ethier
fbf9a4c772 executor: implement streaming stats API
plugins/driver: update driver interface to support streaming stats

client/tr: use streaming stats api

TODO:
 * how to handle errors and closed channel during stats streaming
 * prevent tight loop if Stats(ctx) returns an error

drivers: update drivers TaskStats RPC to handle streaming results

executor: better error handling in stats rpc

docker: better control and error handling of stats rpc

driver: allow stats to return a recoverable error
2019-01-12 12:18:22 -05:00
Alex Dadgar
e3859dc5e5 fix rkt use of executor 2019-01-11 11:36:37 -08:00
Alex Dadgar
270ae48b82 Plugins use parent loggers
This PR fixes various instances of plugins being launched without using
the parent loggers. This meant that logs would not all go to the same
output, break formatting etc.
2019-01-11 11:36:37 -08:00
Mahmood Ali
a99b7655cc drivers/java: use libcontainer executor on java linux 2019-01-10 10:10:40 -05:00
Mahmood Ali
4240d6ac18 tests: ignore _JAVA_OPTIONS line
ignore _JAVA_OPTIONS line in `java -version`, as it's relevant.
2019-01-10 10:10:40 -05:00
Mahmood Ali
f056ef5fa8 Merge pull request #5166 from hashicorp/b-docker-tests-mac
tests: run docker tests in macOS out of box
2019-01-09 13:07:37 -05:00
Mahmood Ali
d0efb37b80 Merge pull request #5162 from hashicorp/f-extract-lxc
Extract LXC from nomad
2019-01-09 13:07:05 -05:00
Mahmood Ali
d1fbd735f3 Merge pull request #5157 from hashicorp/r-drivers-no-cstructs
drivers: avoid referencing client/structs package
2019-01-09 13:06:46 -05:00
Mahmood Ali
2349a01274 Merge pull request #5163 from hashicorp/r-minor-changes-20180108
Fix a panic on node.Deregister fail
2019-01-09 09:56:00 -05:00
Mahmood Ali
7840ab51e2 fix more cases of logging arity errors 2019-01-09 09:22:47 -05:00
Mahmood Ali
76d40947d7 Merge pull request #5159 from hashicorp/r-macos-tests
Fix Travis MacOS job
2019-01-09 08:22:30 -05:00
Mahmood Ali
39fa004f75 tests: run docker tests in macOS out of box
Use `/tmp` as temporary directory for docker driver tests, so tests can
run out of the box without any intervention.

macOS sets tempdir as `/var`, which Docker does not whitelist as a path
that can be bind-mounted.
2019-01-08 14:35:40 -05:00
Mahmood Ali
f8f285248d Merge pull request #5154 from hashicorp/f-revert-exec-devs
drivers/exec: restrict devices exposed to tasks
2019-01-08 12:43:06 -05:00
Mahmood Ali
060588da31 executor: add a comment detailing isolation 2019-01-08 12:10:26 -05:00
Mahmood Ali
b51ecacd23 remove lxc references 2019-01-08 09:28:20 -05:00
Mahmood Ali
34ee0ba6b9 Remove some dead code 2019-01-08 09:11:48 -05:00
Mahmood Ali
800a3522e3 drivers: re-export ResourceUsage structs
Re-export the ResourceUsage structs in drivers package to avoid drivers
directly depending on the internal client/structs package directly.

I attempted moving the structs to drivers, but that caused some import
cycles that was a bit hard to disentagle.  Alternatively, I added an
alias here that's sufficient for our purposes of avoiding external
drivers depend on internal packages, while allowing us to restructure
packages in future without breaking source compatibility.
2019-01-08 09:11:47 -05:00