Commit Graph

1476 Commits

Author SHA1 Message Date
Preetha
0fcfaab646 Merge pull request #4303 from hashicorp/b-docker-client-nil-panic
Add nil check before setting timeout on docker client
2018-05-21 19:34:44 -07:00
Alex Dadgar
df8f367b98 Merge pull request #4282 from hashicorp/f-rotator
Avoid splitting log line across two files
2018-05-21 17:52:13 +00:00
Alex Dadgar
d7fd033959 Merge pull request #4298 from justenwalker/docker-driver-digest-tags
driver/docker: pull image with digest
2018-05-21 17:46:14 +00:00
Justen Walker
ba2d9570e8 driver/docker: add test for dockerImageRef 2018-05-14 14:24:03 -04:00
Justen Walker
f4670f6003 driver/docker: fix up TestParseDockerImage 2018-05-14 14:23:48 -04:00
Justen Walker
44923339a8 driver/docker: fix TestDockerDriver_ForcePull_RepoDigest 2018-05-14 14:23:02 -04:00
Nick Ethier
6e86c4e28c client/driver: gaurd authHelper test from running on windows 2018-05-14 13:46:57 -04:00
Justen Walker
ce63b79449 driver/docker: cleanup parseDockerImage 2018-05-14 11:11:51 -04:00
Justen Walker
e77d8a59eb driver/docker: pull image with digest
GH #4290

Add digest support to the docker driver image config. This commit
factors out some common code to print the repo:tag (dockerImageRef) for
events/logs as well as parsing the image to retreive the repo,tag
(parseDockerImage) so that the results are consistent/sane for both
repo:tag and repo@sha256:... references.

When pulling an image with a digest, the tag is blank and the repo
contains the digest. See:
https://github.com/fsouza/go-dockerclient/blob/master/image_test.go#L471
2018-05-14 10:42:58 -04:00
Preetha Appan
ead6d88aba Add nil check before setting timeout on docker client 2018-05-11 17:09:26 -05:00
Alex Dadgar
80ac8c2a37 Add new line test 2018-05-11 10:52:09 -07:00
Alex Dadgar
8aaf074699 Avoid splitting log line across two files
We attempt to avoid splitting a log line between two files by detecting
if we are near the file size limit and scanning for new lines and only
flushing those.

BenchmarkRotator/1KB-8            300000              5613 ns/op
BenchmarkRotator/2KB-8            200000              8384 ns/op
BenchmarkRotator/4KB-8            100000             14604 ns/op
BenchmarkRotator/8KB-8             50000             25002 ns/op
BenchmarkRotator/16KB-8            30000             47572 ns/op
BenchmarkRotator/32KB-8            20000             92080 ns/op
BenchmarkRotator/64KB-8            10000            165883 ns/op
BenchmarkRotator/128KB-8            5000            294405 ns/op
BenchmarkRotator/256KB-8            2000            572374 ns/op
2018-05-10 15:11:01 -07:00
Alex Dadgar
99182736e8 Benchmark for rotator
BenchmarkRotator/1KB-8            200000              5572 ns/op
BenchmarkRotator/2KB-8            200000              8338 ns/op
BenchmarkRotator/4KB-8            100000             14246 ns/op
BenchmarkRotator/8KB-8             50000             25279 ns/op
BenchmarkRotator/16KB-8            30000             48602 ns/op
BenchmarkRotator/32KB-8            20000             92159 ns/op
BenchmarkRotator/64KB-8            10000            154766 ns/op
BenchmarkRotator/128KB-8            5000            296872 ns/op
BenchmarkRotator/256KB-8            3000            551793 ns/op
2018-05-10 14:15:15 -07:00
Nick Ethier
d3c138e310 client/driver: parse repo instead of attempting to pull repo info 2018-05-09 22:34:25 -04:00
Nick Ethier
46c557c08e client/driver: add test for docker auth helper 2018-05-09 22:33:56 -04:00
Alex Dadgar
9966194730 naming of constants 2018-05-09 16:46:52 -07:00
Nick Ethier
bbb3d74744 client/driver: use correct repo address when using docker-credential helper 2018-05-08 15:17:28 -04:00
Nick Ethier
84926f759a client/driver/env: interpolate empty optional meta params as empty strings 2018-05-07 20:19:51 -04:00
Nick Ethier
fdef36ae38 client/driver: remove unused const 'dockerPullProgressEmitInterval' 2018-05-07 16:24:48 -04:00
Nick Ethier
e324e86632 client/driver: add waiting layer status count to pull progress status msg 2018-05-07 12:18:20 -04:00
Nick Ethier
6337f8e32e client/driver: add seperate handler for emitting pull progress 2018-05-07 12:17:34 -04:00
Nick Ethier
9ea3080343 client/driver: remove pull timeout due to race condition that can lead to unexpected timeouts
If two jobs are pulling the same image simultaneously, which ever starts the pull first will set the pull timeout.
This can lead to a poor UX where the first job requested a short timeout while the second job requested a longer timeout
causing the pull to potentially timeout much sooner than expected by the second job.
2018-05-07 12:18:11 -04:00
Nick Ethier
6fb6ecdff6 client/driver: do accounting on layer pull progress 2018-05-07 12:17:53 -04:00
Nick Ethier
c44db4ce79 client/driver: emit progress to all allocs pulling same image 2018-05-07 12:17:34 -04:00
Nick Ethier
77d57e8276 client/driver: add image pull progress monitoring 2018-05-07 12:17:38 -04:00
Michael Schurter
c79a820042 Merge pull request #4188 from hashicorp/f-rkt-stats
rkt: create parent cgroup to enable stats
2018-04-24 14:54:36 -07:00
Michael Schurter
a3dba1db78 rkt: test Stats() and always run tests
Remove the NOMAD_TEST_RKT flag as a guard for rkt tests. Still require
Linux, root, and rkt to be installed. Only check for rkt installation
once in hopes of speeding up rkt tests a bit.
2018-04-24 11:05:42 -07:00
Javier Palomo Almena
92cbfd0cf3 docker tests: Fix usage of NewDriverContext 2018-04-23 22:51:06 +02:00
Javier Palomo Almena
3e6c3437af DriverContext: Add the TaskGroup and the Job name
Adding this fields to the DriverContext object, will allow us to pass
them to the drivers.

An use case for this, will be to emit tagged metrics in the drivers,
which contain all relevant information:
- Job
- TaskGroup
- Task
- ...

Ref: https://github.com/hashicorp/nomad/pull/4185
2018-04-23 00:15:29 +02:00
Michael Schurter
c06891956d rkt: create parent cgroup to enable stats
Having the Nomad executor create parent cgroups that rkt is launched
within allows the stats collection code used for the exec driver to Just
Work. The only downside is that now the Nomad executor's resource
utilization counts against the cgroups resource limits just as it does
for the exec driver.
2018-04-19 15:14:56 -07:00
Michael Schurter
a4bf901559 run goimports 2018-04-19 11:16:28 -07:00
Michael Schurter
d6de69b6fc Merge pull request #4168 from ninoles/b-2117-windows-group-process
B 2117 windows group process
2018-04-19 11:10:51 -07:00
Michael Schurter
29da24b77b test: build with mock_driver by default
`make release` and `make prerelease` set a `release` tag to disable
enabling the `mock_driver`
2018-04-18 14:45:33 -07:00
Fabien Ninoles
d6cc8895ca Update based on PR request. 2018-04-17 13:43:04 -04:00
Fabien Ninoles
7023b63ffa - Clean up for windows compilation.
- Set CREATE_NEW_PROCESS_GROUP for Windows subprocess.
- Ensure we only kill actual process that need to.
2018-04-14 13:58:42 -04:00
Michael Schurter
1e67780f1e Merge pull request #3572 from emate/master
Create new process group on process startup.
2018-04-13 11:56:38 -07:00
Alex Dadgar
f45b51a138 Driver health detection cleanups
This PR does:

1. Health message based on detection has format "Driver XXX detected"
and "Driver XXX not detected"
2. Set initial health description based on detection status and don't
wait for the first health check.
3. Combine updating attributes on the node, fingerprint and health
checking update for drivers into a single call back.
4. Condensed driver info in `node status` only shows detected drivers
and make the output less wide by removing spaces.
2018-04-12 12:46:40 -07:00
Chelsea Holland Komlo
c6cd78db59 only initialize docker clients if they are nil 2018-04-09 14:13:07 -04:00
Chelsea Holland Komlo
4c1c88a91c refacotoring simplification from code review 2018-04-09 10:34:17 -04:00
Chelsea Holland Komlo
d251199432 group similar functions; update comments
health check timeout should be 1 minute
2018-04-05 16:19:02 -04:00
Chelsea Holland Komlo
dee4fc4555 remove do once block when creating a new docker client
only set cached connections upon no error
2018-04-05 16:19:02 -04:00
Chelsea Holland Komlo
45d09d1ef9 use client with shorter timeouts for health checks 2018-04-05 16:19:02 -04:00
Chelsea Holland Komlo
9092439107 refactor docker clients method to be able to extend to creating new clients 2018-04-05 16:19:02 -04:00
Charlie Voiselle
33e57bf5a3 rkt: logging enhancements (#4044)
* Added extra debug logging; extended timeout; added jitter.

* small log changes

* increase timeout

* remove unneccessary uuid
2018-03-27 17:30:06 -07:00
Alex Dadgar
95a7e1a90a Driver Info output 2018-03-22 17:18:32 -07:00
Michael Schurter
3a7a3f32d5 Merge pull request #4022 from hashicorp/f-more-executor-logging
executor: increase level for helpful log lines
2018-03-22 15:21:20 -07:00
Alex Dadgar
48d637dad1 RPC, FSM, State Store for marking DesiredTransistion
fix build tag
2018-03-21 16:49:48 -07:00
Michael Schurter
91e8fd098f mock_driver: improve Kill() logging 2018-03-21 16:49:48 -07:00
Alex Dadgar
b59bea98b0 Docker driver doesn't return errors but injects into the DriverInfo 2018-03-21 15:15:26 -04:00
Chelsea Holland Komlo
06a306e460 improve comments; update watchDriver 2018-03-21 15:15:26 -04:00