Commit Graph

1563 Commits

Author SHA1 Message Date
Michael Schurter
8cdce8b257 Merge pull request #4894 from hashicorp/f-device-hook
Device hook and devices affect computed node class
2018-11-28 12:10:43 -06:00
Mahmood Ali
90daa8b2d3 Fixes in old lxc driver 2018-11-27 21:40:43 -05:00
Michael Schurter
a13607f2d9 client: properly support hook env vars
The old approach was incomplete. Hook env vars are now:

 * persisted and restored between agent restarts
 * deterministic (LWW if 2 hooks set the same key)
2018-11-27 17:25:33 -08:00
Michael Schurter
926e3dc706 client: test more env key variations 2018-11-15 16:26:25 -08:00
Michael Schurter
26211408a6 client: turn env into nested objects for task configs 2018-11-15 16:25:57 -08:00
Michael Schurter
43b359914b client: interpolate driver configurations
Also add missing SetDriverNetwork calls.
2018-11-15 16:25:57 -08:00
Danielle Tomlinson
f16d96bdd8 Merge pull request #4869 from hashicorp/b-executor-stdout
executor: Fix stdout stderr copy/paste
2018-11-13 19:22:37 -08:00
Mahmood Ali
179cdc6277 Address review comments 2018-11-13 10:21:40 -05:00
Mahmood Ali
5fe433efe7 avoid setting resource limit on rkt command
Was accidentally modified in 5b14d24bf4 .
2018-11-13 10:21:40 -05:00
Mahmood Ali
9933f4a45c Fix docker log fetching in tests
We no longer use syslog for tracking logs so tracking them explicitly
here
2018-11-13 10:21:40 -05:00
Mahmood Ali
9d8a71dc44 killing should be done with wait client
Incidentally changed in 5b14d24bf4
2018-11-13 10:21:40 -05:00
Mahmood Ali
73077e36fe Update Docker name parsing lookup
`ParseNamed` function changed in e9f3f2cfee
where became `ParsedNormalizedName` with extra checks.
2018-11-13 10:21:40 -05:00
Danielle Tomlinson
04577f7e15 executor: Fix stdout stderr copy/paste 2018-11-12 22:08:04 -08:00
Nick Ethier
4b08ef0534 Merge pull request #4765 from jippi/increase-line-scan-limit
fix: increase log rotator line scan limit
2018-10-29 18:46:30 -07:00
Preetha Appan
4231dc4729 Stat path to binary to handle raw exec driver interpolated binary path 2018-10-26 17:24:05 -05:00
Preetha Appan
af3a62e750 Fix test linting 2018-10-26 10:30:12 -05:00
Nick Ethier
7f5046849e drivers/exec: add exec implementation 2018-10-16 22:45:28 -04:00
Alex Dadgar
3c0b073513 compile on windows 2018-10-16 16:56:56 -07:00
Alex Dadgar
7b7cb382dc more test fixes 2018-10-16 16:56:56 -07:00
Michael Schurter
4d1a1ac5bb tests: test logs endpoint against pending task
Although the really exciting change is making WaitForRunning return the
allocations that it started. This should cut down test boilerplate
significantly.
2018-10-16 16:56:55 -07:00
Michael Schurter
349b827d7d mock_driver: close waitCh after exiting
mock_driver wasn't behaving like other driver handles.
2018-10-16 16:56:55 -07:00
Nick Ethier
c9f0d2e0b4 driver/raw_exec: port existing raw_exec tests and add some testing utilities 2018-10-16 16:53:31 -07:00
Nick Ethier
650ac5a83e driver/raw_exec: more tests and bug fixes
added wrapper struct for plugin.ReattachConfig to better handle serialization
2018-10-16 16:53:31 -07:00
Nick Ethier
e2bf0a388e clientv2: base driver plugin (#4671)
Driver plugin framework to facilitate development of driver plugins.

Implementing plugins only need to implement the DriverPlugin interface.
The framework proxies this interface to the go-plugin GRPC interface generated
from the driver.proto spec.

A testing harness is provided to allow implementing drivers to test the full
lifecycle of the driver plugin. An example use:

func TestMyDriver(t *testing.T) {
    harness := NewDriverHarness(t, &MyDiverPlugin{})
    // The harness implements the DriverPlugin interface and can be used as such
    taskHandle, err := harness.StartTask(...)
}
2018-10-16 16:53:31 -07:00
Nick Ethier
5b14d24bf4 executor v2 (#4656)
* client/executor: refactor client to remove interpolation

* executor: POC libcontainer based executor

* vendor: use hashicorp libcontainer fork

* vendor: add libcontainer/nsenter dep

* executor: updated executor interface to simplify operations

* executor: implement logging pipe

* logmon: new logmon plugin to manage task logs

* driver/executor: use logmon for log management

* executor: fix tests and windows build

* executor: fix logging key names

* executor: fix test failures

* executor: add config field to toggle between using libcontainer and standard executors

* logmon: use discover utility to discover nomad executable

* executor: only call libcontainer-shim on main in linux

* logmon: use seperate path configs for stdout/stderr fifos

* executor: windows fixes

* executor: created reusable pid stats collection utility that can be used in an executor

* executor: update fifo.Open calls

* executor: fix build

* remove executor from docker driver

* executor: Shutdown func to kill and cleanup executor and its children

* executor: move linux specific universal executor funcs to seperate file

* move logmon initialization to a task runner hook

* client: doc fixes and renaming from code review


* taskrunner: use shared config struct for logmon fifo fields

* taskrunner: logmon only needs to be started once per task
2018-10-16 16:53:31 -07:00
Nick Ethier
e9f3f2cfee Update runc/libcontainer and friends (#4655)
* vendor: bump libcontainer and docker to remove Sirupsen imports

* vendor: fix bad vendoring of archive package

* vendor: fix api changes to cgroups in executor

* vendor: fix docker api changes

* vendor: update github.com/Azure/go-ansiterm to use non capitalized logrus import
2018-10-16 16:53:30 -07:00
Michael Schurter
051c5ea891 keep forgetting lxc 2018-10-16 16:53:30 -07:00
Michael Schurter
cea2b879e4 Add Network method to Handle interface
Should probably be moved to an Inspect method in the Driver Plugin world
2018-10-16 16:53:30 -07:00
Michael Schurter
fae947419d tr: add validate task hook 2018-10-16 16:53:29 -07:00
Alex Dadgar
deede95117 Initial V2 alloc runner 2018-10-16 16:53:28 -07:00
Alex Dadgar
5e67b37aad use int64 2018-10-16 15:34:32 -07:00
Preetha Appan
3ca71ae935 Change CPU/Disk/MemoryMB to int everywhere in new resource structs 2018-10-16 16:21:42 -05:00
Christian Winther
5314bfcd34 fix: increase log rotator line scan limit
In case where gelf/json logging is used, its fairly easy to exceed the 16k limit, resulting in json output being cut up into multiple strings

the result is invalid json lines which can create all kind of badness in the logging server

This fixes https://github.com/hashicorp/nomad/issues/4699

Signed-off-by: Christian Winther <jippignu@gmail.com>
2018-10-09 18:57:18 +02:00
Alex Dadgar
0f2f4797cb fixing tests 2018-10-04 14:26:19 -07:00
Alex Dadgar
da0bec03c1 Fix make check errors 2018-09-04 16:03:52 -07:00
Alex Dadgar
383251f53b Fix kill timeout exceeding 5m on Docker driver
Fixes an issue where the Docker API client would timeout before the kill
timeout was hit.
2018-08-17 16:01:09 -07:00
Alex Dadgar
33298ab14e Merge pull request #4535 from hashicorp/f-keep-docker-container-0.8.4
Option to prevent removal of container on exit
2018-07-26 11:11:22 -07:00
Charlie Voiselle
61bec1e212 Option to prevent removal of container on exit 2018-07-26 11:10:48 -07:00
Michael Schurter
b2d429400a Merge pull request #4462 from omame/omame/cpu_cfs_period
Add support for specifying cpu_cfs_period in the Docker driver
2018-07-25 09:34:38 -07:00
Daniele Valeriani
a4771a49ab Add test for cpu_cfs_period 2018-07-16 22:43:34 +02:00
Michael Schurter
6014a5089c rkt: revert to redis 3.2 to favor stability 2018-07-09 16:15:32 -07:00
Michael Schurter
734602e16a rkt: speed up tests
Disable networking when it's not needed and improve failure message for
UserGroup test by including the full ps output on failure.
2018-07-09 14:02:27 -07:00
Michael Schurter
0258c141bb rkt: skip retrieving network information when net=none
Even when net=none we would attempt to retrieve network information from
rkt which would spew useless log lines such as:

```
testlog.go:30: 20:37:31.409209 [DEBUG] driver.rkt: failed getting network info for pod UUID 8303cfe6-0c10-4288-84f5-cb79ad6dbf1c attempt 2: no networks found. Sleeping for 970ms
```

It would also delay tests for ~60s during the network information retry
period.

So skip this when net=none. It's unlikely anyone actually uses net=none
outside of tests, so I doubt anyone will notice this change.

Official docs:
https://coreos.com/rkt/docs/latest/networking/overview.html#no-loopback-only-networking
2018-07-09 13:44:43 -07:00
Michael Schurter
b0269d5183 tests: make alloc id consistent in helper
It worked, but the old code used a different alloc id for the path than
the actual alloc! Use the same alloc id everywhere to prevent confusing
test output.
2018-07-09 13:37:35 -07:00
Michael Schurter
2795883e23 rkt: fix failing TestRktDriver_UserGroup test
Started failing due to the docker redis image switching from Debian
jessie to stretch:
53f8680550 (diff-acff46b161a3b7d6ed01ba79a032acc9)

Switched from Debian based image to Alpine to get a working `ps` command
again (albeit busybox's stripped down implementation)
2018-07-09 12:19:02 -07:00
Daniele Valeriani
f47ac0d430 Validate the value of cpu_cfs_period 2018-07-02 22:30:22 +02:00
Daniele Valeriani
f94e1c3d2e Remove an unnecessary conversion 2018-07-02 17:47:23 +02:00
Daniele Valeriani
dd7699abdb Add support for specifying cpu_cfs_period in the Docker driver 2018-07-02 16:37:04 +02:00
Alex Dadgar
98c7abe541 Tests only use testlog package logger 2018-06-13 15:40:56 -07:00
Nick Ethier
d3e6c10727 Merge pull request #4403 from hashicorp/b-fix-dispatched-optional-meta
Fix dispatched optional meta correctly
2018-06-11 16:17:14 -04:00