Commit Graph

422 Commits

Author SHA1 Message Date
Mahmood Ali
2cc2e60ded update comment 2019-06-11 13:00:26 -04:00
Mahmood Ali
c72bf13f8a exec: use an independent name=systemd cgroup path
We aim for containers to be part of a new cgroups hierarchy independent
from nomad agent.  However, we've been setting a relative path as
libcontainer `cfg.Cgroups.Path`, which makes libcontainer concatinate
the executor process cgroup with passed cgroup, as set in [1].

By setting an absolute path, we ensure that all cgroups subsystem
(including `name=systemd` get a dedicated one).  This matches behavior
in Nomad 0.8, and behavior of how Docker and OCI sets CgroupsPath[2]

Fixes #5736

[1] d7edf9b2e4/vendor/github.com/opencontainers/runc/libcontainer/cgroups/fs/apply_raw.go (L326-L340)
[2] 238f8eaa31/vendor/github.com/containerd/containerd/oci/spec.go (L229)
2019-06-10 22:00:12 -04:00
Chris Baker
7b6d233617 docker/driver: downgraded log level for error in DestroyTask 2019-06-03 21:21:32 +00:00
Chris Baker
262c863a8b drivers/docker: modify container/image cleanup to be robust to containers removed out of band 2019-06-03 19:52:28 +00:00
Chris Baker
3b827703fb docker/tests:
- modified tests to cleanup now that RemoveContainer isn't in StartTask
- fix some broken tests by removing docker images/containers before test
2019-06-03 19:05:08 +00:00
Chris Baker
3a96683131 docker: DestroyTask was not cleaning up Docker images because it was erroring early due to an attempt to inspect an image that had already been removed 2019-06-03 19:04:27 +00:00
Mahmood Ali
6217d50803 Fix test comparisons 2019-05-24 21:38:22 -05:00
Mahmood Ali
a1414bd360 Test for expected capabilities specifically 2019-05-24 16:07:05 -05:00
Mahmood Ali
e855738e0c use /bin/bash 2019-05-24 14:50:23 -04:00
Mahmood Ali
1a6454d242 special case root capabilities 2019-05-24 14:10:10 -04:00
Mahmood Ali
3e1b136929 tests: Fix binary dir permissions 2019-05-24 11:31:12 -04:00
Mahmood Ali
67188714a3 fix 2019-05-20 15:30:07 -04:00
Mahmood Ali
82611af925 drivers/exec: Restore 0.8 capabilities
Nomad 0.9 incidentally set effective capabilities that is higher than
what's expected of a `nobody` process, and what's set in 0.8.

This change restores the capabilities to ones used in Nomad 0.9.
2019-05-20 13:11:29 -04:00
Lang Martin
568a120e7b Merge pull request #5649 from hashicorp/b-lookup-exe-chroot
lookup executables inside chroot
2019-05-17 15:07:41 -04:00
Michael Schurter
4b854cc557 drivers/mock: implement InspectTask 2019-05-14 10:53:27 -07:00
Mahmood Ali
494642b11c typo: "atleast" -> "at least" 2019-05-13 10:01:19 -04:00
Mahmood Ali
bfac34d9c5 implement nomad exec for rkt
Implement the streaming exec handler for the rkt driver
2019-05-12 18:59:00 -04:00
Mahmood Ali
7f76aedfae use pty/tty terminology similar to github.com/kr/pty 2019-05-10 19:17:14 -04:00
Mahmood Ali
74e5e20c0b drivers: implement streaming exec for executor based drivers
These simply delegate call to backend executor.
2019-05-10 19:17:14 -04:00
Mahmood Ali
976bfbc41a executors: implement streaming exec
Implements streamign exec handling in both executors (i.e. universal and
libcontainer).

For creation of TTY, some incidental complexity leaked in.  The universal
executor uses github.com/kr/pty for creation of TTYs.

On the other hand, libcontainer expects a console socket and for libcontainer to
create the underlying console object on process start.  The caller can then use
`libcontainer.utils.RecvFd()` to get tty master end.

I chose github.com/kr/pty for managing TTYs here.  I tried
`github.com/containerd/console` package (which is already imported), but the
package did not work as expected on macOS.
2019-05-10 19:17:14 -04:00
Mahmood Ali
efc4249f85 executor: scaffolding for executor grpc handling
Prepare executor to handle streaming exec API calls that reuse drivers protobuf
structs.
2019-05-10 19:17:14 -04:00
Lang Martin
a9da4bac11 executor_linux only do path resolution in the taskDir, not local
split out lookPathIn to show it's similarity to exec.LookPath
2019-05-10 11:33:35 -04:00
Mahmood Ali
67160a6302 drivers/docker: implement streaming exec 2019-05-09 16:49:08 -04:00
Mahmood Ali
cb4ad3fb45 drivers/mock: implement nomad exec interface 2019-05-09 16:49:08 -04:00
Lang Martin
60735deb7b executor_linux_test call lookupTaskBin with an ExecCommand 2019-05-08 10:01:51 -04:00
Lang Martin
0c4a45fa16 executor_linux pass the command to lookupTaskBin to get path 2019-05-08 10:01:20 -04:00
Lang Martin
9688710c10 executor/* Launch log at top of Launch is more explicit, trace 2019-05-07 17:01:05 -04:00
Lang Martin
538f387d8d move lookupTaskBin to executor_linux, for os dependency clarity 2019-05-07 16:58:27 -04:00
Lang Martin
2ea860a087 driver_test leave cat in the test, but add cat to the chroot 2019-05-07 16:14:01 -04:00
Lang Martin
bc5eaf6cdb executor_test cleanup old lookupBin tests 2019-05-04 10:21:59 -04:00
Lang Martin
8c638ada5a driver_test StartWait task calls a program inside it's chroot 2019-05-03 16:34:31 -04:00
Lang Martin
8a7b5c6830 executor lookupTaskBin also does PATH expansion, anchored in taskDIR 2019-05-03 16:22:09 -04:00
Lang Martin
512bc52af5 executor_linux_test test PATH lookup inside the container 2019-05-03 16:21:58 -04:00
Lang Martin
7a2fdf7a2d executor and executor_linux debug launch prep and process start 2019-05-03 14:42:57 -04:00
Lang Martin
103d37d4f9 executor_linux_test new TestExecutor_EscapeContainer 2019-05-03 14:38:42 -04:00
Lang Martin
c16f82bb9a executor_test test for more edges of lookupBin behavior 2019-05-03 11:55:19 -04:00
Lang Martin
83930dcb9c executor_linux call new lookupTaskBin 2019-05-03 11:55:19 -04:00
Lang Martin
4538014cc3 executor split up lookupBin 2019-05-03 11:55:19 -04:00
Mahmood Ali
e79ce1f9d0 drivers/mock: extract command related operations
Extract command parsing and execution mocking into a separate struct.  Also,
allow mocking of different fs_isolation for testing.
2019-04-30 14:02:16 -04:00
Mahmood Ali
fce6564ce4 driver/docker: Support volumes field in Windows
Support Docker `volumes` field in Windows.  Previously, volumes parser
assumed some Unix-ism (e.g. didn't expect `:` in mount paths).
Here, we use the Docker parser to identify host and container paths.

Docker parsers use different validation logic from our previous unix
implementation: Docker parser accepts single path as a volume entry
(parsing it as a container path with auto-created volume) and enforces
additional checks (e.g. validity of mode).  Thereforce, I opted to use
Docker parser only for Windows, and keep Nomad's linux parser to
preserve current behavior.
2019-04-25 09:02:44 -04:00
Mahmood Ali
ce8a8a5326 driver/docker: collect tty container logs
Fixes https://github.com/hashicorp/nomad/issues/5475

When container is a tty container, we need to get raw terminal output
without any additional processing.
2019-04-24 22:01:51 -04:00
Mahmood Ali
6747195682 comment on using init() for libcontainer handling 2019-04-19 09:49:04 -04:00
Mahmood Ali
9bf54eae97 comment what refer to 2019-04-19 09:49:04 -04:00
Mahmood Ali
b6af5c9dca Move libcontainer helper to executor package 2019-04-19 09:49:04 -04:00
Danielle Lancashire
ccce364cbd Switch to pre-0.9 behaviour for handling volumes
In Nomad 0.9, we made volume driver handling the same for `""`, and
`"local"` volumes. Prior to Nomad 0.9 however these had slightly different
behaviour for relative paths and named volumes.

Prior to 0.9 the empty string would expand relative paths within the task
dir, and `"local"` volumes that are not absolute paths would be treated
as docker named volumes.

This commit reverts to the previous behaviour as follows:

| Nomad Version | Driver  |   Volume Spec    | Behaviour                 |
|-------------------------------------------------------------------------
| all           | ""      | testing:/testing | allocdir/testing          |
| 0.8.7         | "local" | testing:/testing | "testing" as named volume |
| 0.9.0         | "local" | testing:/testing | allocdir/testing          |
| 0.9.1         | "local" | testing:/testing | "testing" as named volume |
2019-04-18 14:28:45 +02:00
Mahmood Ali
96a54cbbd3 locking and opening streams in goroutine comment 2019-04-16 11:02:19 -04:00
Mahmood Ali
316a161ce8 open fifo on background goroutine 2019-04-15 21:20:09 -04:00
Mahmood Ali
60cfa194e4 Merge pull request #5537 from hashicorp/b-nonvidia-flag
Allow compiling without nvidia integration
2019-04-10 13:40:55 -04:00
Mahmood Ali
a09e3bf1a1 Allow compiling without nvidia integration
nvidia library use of dynamic library seems to conflict with alpine and
musl based OSes.  This adds a `nonvidia` tag to allow compiling nomad
for alpine images.

The nomad releases currently only support glibc based OS environments,
so we default to compiling with nvidia.
2019-04-10 09:19:12 -04:00
Mahmood Ali
43faf1176c ci: move docker unix-y test
Fix AppVeyor failing builds, by moving docker image url test to run on unix
systems only.  The used paused image is a linux image only, not
available on Windows.
2019-04-09 19:59:58 -04:00