Commit Graph

455 Commits

Author SHA1 Message Date
Michael Schurter
dca40b5737 Merge pull request #5999 from Iqoqo/use-default-network-for-docker
driver/docker: use default network mode
2019-08-01 09:58:12 -07:00
Ilya Guterman
7beee87618 driver/docker: support unix destination mount path in windows
This reverts commit a6c96eade5.
2019-08-01 19:54:08 +03:00
Ilya Guterman
d4919cebdc driver/docker: use default network mode
fallback to docker default network mode instead of explicit bridge for linux
or nat for windows
2019-07-31 21:07:46 +03:00
Nick Ethier
cf65bbe47c docker: allow configuration of infra image 2019-07-31 01:04:07 -04:00
Nick Ethier
1ff85f09f3 executor: cleanup netns handling in executor 2019-07-31 01:04:05 -04:00
Nick Ethier
d28d865100 executor: support network namespacing on universal executor 2019-07-31 01:03:58 -04:00
Nick Ethier
4fdb0dab1c docker: add nil check on network isolation spec 2019-07-31 01:03:21 -04:00
Nick Ethier
c8c1ad748e docker: fix driver test from changed func args 2019-07-31 01:03:20 -04:00
Nick Ethier
67ea2e9305 docker: add additional commens 2019-07-31 01:03:20 -04:00
Nick Ethier
51b69601ab docker: support shared network namespaces 2019-07-31 01:03:20 -04:00
Nick Ethier
e26192ad49 Driver networking support
Adds support for passing network isolation config into drivers and
implements support in the rawexec driver as a proof of concept
2019-07-31 01:03:20 -04:00
Nick Ethier
da3978b377 plugins/driver: make DriverNetworkManager interface optional 2019-07-31 01:03:19 -04:00
Nick Ethier
4a8a96fa1a ar: initial driver based network management 2019-07-31 01:03:17 -04:00
Michael Schurter
a6c96eade5 Revert "driver/docker: support unix destination mount path in windows" 2019-07-29 09:09:54 -07:00
Michael Schurter
164d166507 Merge pull request #6001 from Iqoqo/allow-unix-container-path-in-lcow
driver/docker: support unix destination mount path in windows
2019-07-29 09:09:41 -07:00
Lang Martin
c7cd018655 executor_universal_linux log a link to the docs on cgroup error 2019-07-24 12:37:33 -04:00
Ilya Guterman
e30a24edf1 driver/docker: support unix destination mount path in windows 2019-07-24 11:57:28 +03:00
Lang Martin
cab04997f0 executor_universal_linux raw_exec cgroup failure is not fatal 2019-07-22 15:16:36 -04:00
Lang Martin
7bd881cbf7 default e.getAllPids in executor_basic 2019-07-18 10:57:27 -04:00
Lang Martin
ab3e6259d0 executor_unix and _windows stub getAllPids ByScanning 2019-07-17 17:34:06 -04:00
Lang Martin
1a9c598fc2 executor_universal_linux getAllPids chooses cgroup when available 2019-07-17 17:33:55 -04:00
Lang Martin
3834616691 executor use e.getAllPids() 2019-07-17 17:33:11 -04:00
Lang Martin
d3ef456bd7 resource_container_linux new getAllPidsByCgroup 2019-07-17 17:31:36 -04:00
Lang Martin
412997f566 pid_collector getAllPids -> getAllPidsByScanning 2019-07-17 17:31:20 -04:00
Jasmine Dahilig
3fdb3cbe33 Merge pull request #5846 from hashicorp/f-docker-log-constraints
add log rotation to docker driver log defaults
2019-07-03 10:17:19 -07:00
Jasmine Dahilig
8e7a2d0765 default to json-file log rotation for docker driver 2019-07-03 09:04:45 -07:00
Mahmood Ali
858e98cea0 Merge pull request #5811 from cloudbuy/b-win32-volume-split
lift code from docker/volume/mounts for splitting windows volumes
2019-06-18 21:19:15 -04:00
Mahmood Ali
8fb9d25041 comment on use of init() for plugin handlers 2019-06-18 20:54:55 -04:00
Damien Churchill
0cce6977d7 run new file through goimports 2019-06-18 08:35:25 +01:00
Damien Churchill
cb8a5e4caa run gofmt over the new file 2019-06-18 08:35:25 +01:00
Damien Churchill
960f898dff drivers/docker: move lifted code out to separate file and link the source & license 2019-06-18 08:35:25 +01:00
Damien Churchill
4f04c03c27 lift code from docker/volume/mounts for splitting windows volumes
Using the API as provided from the `mounts` package imposes validation
on the `src:dest` which shouldn't be performed at this time. To workaround
that lift the internal code from that library required to only perform
the split.
2019-06-18 08:35:25 +01:00
Mahmood Ali
eeaa95ddf9 Use init to handle plugin invocation
Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI
commands to be handled by command CLI framework.  Plugin launchers use
`discover.NomadBinary()` to identify the binary and start it.

This has few downsides: The trivial one is that when running tests, one
must re-compile the nomad binary as the tests need to invoke the nomad
executable to start plugin.  This is frequently overlooked, resulting in
puzzlement.

The more significant issue with `executor` in particular is in relation
to external driver:

* Plugin must identify the path of invoking nomad binary, which is not
trivial; `discvoer.NomadBinary()` now returns the path to the plugin
rather than to nomad, preventing external drivers from launching
executors.

* The external driver may get a different version of executor than it
expects (specially if we make a binary incompatible change in future).

This commit addresses both downside by having the plugin invocation
handling through an `init()` call, similar to how libcontainer init
handler is done in [1] and recommened by libcontainer [2].  `init()`
will be invoked and handled properly in tests and external drivers.

For external drivers, this change will cause external drivers to launch
the executor that's compiled against.

There a are a couple of downsides to this approach:
* These specific packages (i.e executor, logmon, and dockerlog) need to
be careful in use of `init()`, package initializers.  Must avoid having
command execution rely on any other init in the package.  I prefixed
files with `z_` (golang processes files in lexical order), but ensured
we don't depend on order.
* The command handling is spread in multiple packages making it a bit
less obvious how plugin starts are handled.

[1] drivers/shared/executor/libcontainer_nsenter_linux.go
[2] eb4aeed24f/libcontainer (using-libcontainer)
2019-06-13 16:48:01 -04:00
Mahmood Ali
2cc2e60ded update comment 2019-06-11 13:00:26 -04:00
Mahmood Ali
c72bf13f8a exec: use an independent name=systemd cgroup path
We aim for containers to be part of a new cgroups hierarchy independent
from nomad agent.  However, we've been setting a relative path as
libcontainer `cfg.Cgroups.Path`, which makes libcontainer concatinate
the executor process cgroup with passed cgroup, as set in [1].

By setting an absolute path, we ensure that all cgroups subsystem
(including `name=systemd` get a dedicated one).  This matches behavior
in Nomad 0.8, and behavior of how Docker and OCI sets CgroupsPath[2]

Fixes #5736

[1] d7edf9b2e4/vendor/github.com/opencontainers/runc/libcontainer/cgroups/fs/apply_raw.go (L326-L340)
[2] 238f8eaa31/vendor/github.com/containerd/containerd/oci/spec.go (L229)
2019-06-10 22:00:12 -04:00
Chris Baker
7b6d233617 docker/driver: downgraded log level for error in DestroyTask 2019-06-03 21:21:32 +00:00
Chris Baker
262c863a8b drivers/docker: modify container/image cleanup to be robust to containers removed out of band 2019-06-03 19:52:28 +00:00
Chris Baker
3b827703fb docker/tests:
- modified tests to cleanup now that RemoveContainer isn't in StartTask
- fix some broken tests by removing docker images/containers before test
2019-06-03 19:05:08 +00:00
Chris Baker
3a96683131 docker: DestroyTask was not cleaning up Docker images because it was erroring early due to an attempt to inspect an image that had already been removed 2019-06-03 19:04:27 +00:00
Mahmood Ali
6217d50803 Fix test comparisons 2019-05-24 21:38:22 -05:00
Mahmood Ali
a1414bd360 Test for expected capabilities specifically 2019-05-24 16:07:05 -05:00
Mahmood Ali
e855738e0c use /bin/bash 2019-05-24 14:50:23 -04:00
Mahmood Ali
1a6454d242 special case root capabilities 2019-05-24 14:10:10 -04:00
Mahmood Ali
3e1b136929 tests: Fix binary dir permissions 2019-05-24 11:31:12 -04:00
Mahmood Ali
67188714a3 fix 2019-05-20 15:30:07 -04:00
Mahmood Ali
82611af925 drivers/exec: Restore 0.8 capabilities
Nomad 0.9 incidentally set effective capabilities that is higher than
what's expected of a `nobody` process, and what's set in 0.8.

This change restores the capabilities to ones used in Nomad 0.9.
2019-05-20 13:11:29 -04:00
Lang Martin
568a120e7b Merge pull request #5649 from hashicorp/b-lookup-exe-chroot
lookup executables inside chroot
2019-05-17 15:07:41 -04:00
Michael Schurter
4b854cc557 drivers/mock: implement InspectTask 2019-05-14 10:53:27 -07:00
Mahmood Ali
494642b11c typo: "atleast" -> "at least" 2019-05-13 10:01:19 -04:00
Mahmood Ali
bfac34d9c5 implement nomad exec for rkt
Implement the streaming exec handler for the rkt driver
2019-05-12 18:59:00 -04:00