Commit Graph

111 Commits

Author SHA1 Message Date
Alex Dadgar
b9f36134dc move catalog + grpcutils 2019-01-22 15:11:57 -08:00
Nick Ethier
f3ab65947e drivers: regen proto 2019-01-18 18:53:45 -05:00
Nick Ethier
994c66f7d7 drivers: use consts for task handle version 2019-01-18 18:31:01 -05:00
Nick Ethier
7c440aa437 drivers: fix test 2019-01-18 18:31:01 -05:00
Nick Ethier
2118d94b2d executor: add pre 0.9 client and wrapper 2019-01-18 18:30:58 -05:00
Mahmood Ali
b5c20aa50b Track Basic Memory Usage as reported by cgroups
Track current memory usage, `memory.usage_in_bytes`, in addition to
`memory.max_memory_usage_in_bytes` and friends.  This number is closer
what Docker reports.

Related to https://github.com/hashicorp/nomad/issues/5165 .
2019-01-14 18:47:52 -05:00
Nick Ethier
72a4685534 drivers: plumb grpc client logger 2019-01-12 12:18:23 -05:00
Nick Ethier
9904463da2 executor: fix failing stats related test 2019-01-12 12:18:23 -05:00
Nick Ethier
f6af1d4d04 docker: add test for stats collection 2019-01-12 12:18:22 -05:00
Nick Ethier
fbf9a4c772 executor: implement streaming stats API
plugins/driver: update driver interface to support streaming stats

client/tr: use streaming stats api

TODO:
 * how to handle errors and closed channel during stats streaming
 * prevent tight loop if Stats(ctx) returns an error

drivers: update drivers TaskStats RPC to handle streaming results

executor: better error handling in stats rpc

docker: better control and error handling of stats rpc

driver: allow stats to return a recoverable error
2019-01-12 12:18:22 -05:00
Mahmood Ali
d1fbd735f3 Merge pull request #5157 from hashicorp/r-drivers-no-cstructs
drivers: avoid referencing client/structs package
2019-01-09 13:06:46 -05:00
Mahmood Ali
855e3684e3 fixup! remove unused field 2019-01-08 12:58:12 -05:00
Mahmood Ali
b0a3f6b491 remove unused field 2019-01-08 12:19:44 -05:00
Mahmood Ali
34ee0ba6b9 Remove some dead code 2019-01-08 09:11:48 -05:00
Mahmood Ali
800a3522e3 drivers: re-export ResourceUsage structs
Re-export the ResourceUsage structs in drivers package to avoid drivers
directly depending on the internal client/structs package directly.

I attempted moving the structs to drivers, but that caused some import
cycles that was a bit hard to disentagle.  Alternatively, I added an
alias here that's sufficient for our purposes of avoiding external
drivers depend on internal packages, while allowing us to restructure
packages in future without breaking source compatibility.
2019-01-08 09:11:47 -05:00
Mahmood Ali
c0162fab35 move cstructs.DeviceNetwork to drivers pkg 2019-01-08 09:11:47 -05:00
Mahmood Ali
694e3010c2 use drivers.FSIsolation 2019-01-08 09:11:47 -05:00
Danielle Tomlinson
994057abf4 drivers: Add internal interface for Shutdown
This allows us to correctly terminate internal state during runs of the
nomad test suite, e.g closing eventer contexts correctly.
2019-01-08 13:48:49 +01:00
Alex Dadgar
4e7da3ea4e regenerate protos 2019-01-07 14:49:40 -08:00
Alex Dadgar
437f03d877 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar
ffadab1b20 remove nil logger 2019-01-07 14:48:01 -08:00
Preetha Appan
26594aa31e Standardize driver health description messages for all drivers 2019-01-06 22:06:38 -06:00
Danielle Tomlinson
749eab7257 chore: Fix environement->environment typo 2019-01-03 13:31:26 +01:00
Danielle Tomlinson
26b5ad4532 Merge pull request #5041 from hashicorp/dani/b-driver-healt
drivers: Cleanup root user fingerprinting
2019-01-03 13:16:28 +01:00
Danielle Tomlinson
4798dacafd plugins: Add consistent message for requires root 2018-12-20 12:54:01 +01:00
Alex Dadgar
99df4c98c7 Store device envs separately and pass to drivers 2018-12-19 14:23:09 -08:00
Alex Dadgar
9a017adfc9 proto 2018-12-19 13:54:19 -08:00
Nick Ethier
6951ca487d drivermanager: use allocID and task name to route task events 2018-12-18 23:01:51 -05:00
Alex Dadgar
ed4f8eac6e Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Alex Dadgar
da6925bfc1 utilities 2018-12-18 15:48:52 -08:00
Alex Dadgar
e1cf3ac69e protos 2018-12-18 15:48:52 -08:00
Nick Ethier
81ba18d74a executor: encode mounts and devices correctly when using grpc 2018-12-15 00:08:23 -05:00
Nick Ethier
8a344412e8 Merge branch 'master' into f-grpc-executor
* master: (71 commits)
  Fix output of 'nomad deployment fail' with no arg
  Always create a running allocation when testing task state
  tests: ensure exec tests pass valid task resources (#4992)
  some changes for more idiomatic code
  fix iops related tests
  fixed bug in loop delay
  gofmt
  improved code for readability
  client: updateAlloc release lock after read
  fixup! device attributes in `nomad node status -verbose`
  drivers/exec: support device binds and mounts
  fix iops bug and increase test matrix coverage
  tests: tag image explicitly
  changelog
  ci: install lxc-templates explicitly
  tests: skip checking rdma cgroup
  ci: use Ubuntu 16.04 (Xenial) in TravisCI
  client: update driver info on new fingerprint
  drivers/docker: enforce volumes.enabled (#4983)
  client: Style: use fluent style for building loggers
  ...
2018-12-13 14:41:09 -05:00
Nick Ethier
a6cb63a964 executor: don't drop errors when configuring libcontainer cfg, add nil check on resources 2018-12-07 14:03:42 -05:00
Nick Ethier
224c68860d executor: use drivers.Resources as resource model 2018-12-06 21:22:02 -05:00
Nick Ethier
13b582fcfb executor: merge plugin shim with executor package 2018-12-06 21:13:45 -05:00
Nick Ethier
0087a51a7a executor: remove structs package 2018-12-06 20:54:14 -05:00
Alex Dadgar
0953d913ed Deprecate IOPS
IOPS have been modelled as a resource since Nomad 0.1 but has never
actually been detected and there is no plan in the short term to add
detection. This is because IOPS is a bit simplistic of a unit to define
the performance requirements from the underlying storage system. In its
current state it adds unnecessary confusion and can be removed without
impacting any users. This PR leaves IOPS defined at the jobspec parsing
level and in the api/ resources since these are the two public uses of
the field. These should be considered deprecated and only exist to allow
users to stop using them during the Nomad 0.9.x release. In the future,
there should be no expectation that the field will exist.
2018-12-06 15:09:26 -08:00
Nick Ethier
467930f650 executor: use grpc instead of netrpc as plugin protocol
* Added protobuf spec for executor
 * Seperated executor structs into their own package
2018-12-05 11:03:56 -05:00
Danielle Tomlinson
113205318c plugins: Add support for serving driver plugins 2018-12-01 17:30:54 +01:00
Danielle Tomlinson
1c98fdf935 plugins: Move driver testing support to subpackage
this allows us to drop a cyclical import, but is subobptimal as it
requires BaseDriver tests to move. This falls firmly into the realm of
being a hack. Alternatives welcome.
2018-12-01 17:29:39 +01:00
Danielle Tomlinson
03db4cf82d client: Rename drivers/shared/env => client/taskenv 2018-11-30 12:18:39 +01:00
Danielle Tomlinson
17d14d7f52 executors: Unify go-plugin handshake 2018-11-30 10:59:23 +01:00
Danielle Tomlinson
5262e42d62 fixup: executorplugin: fix rkt build 2018-11-30 10:47:08 +01:00
Danielle Tomlinson
75d333fe02 client: Move executor plugins into own package 2018-11-30 10:46:13 +01:00
Danielle Tomlinson
23197ec6b4 drivers: Create drivers/shared/structs
This creates a drivers/shared/structs package and moves the buffer size
checks into it.
2018-11-30 10:46:13 +01:00
Danielle Tomlinson
d2136e0aa7 drivers: Move client/drivers/executor to drivers/shared/executor 2018-11-30 10:46:13 +01:00
Danielle Tomlinson
6756ffd052 drivers: Move client/drivers/env to drivers/shared/env
As part of deprecating legacy drivers, we're moving the env package to a
new drivers/shared tree, as it is used by the modern docker and rkt
driver packages, and is useful for 3rd party plugins.
2018-11-30 10:46:13 +01:00
Chris Baker
6f2502e803 Merge pull request #4932 from hashicorp/b-1172-rkt-env-vars
change to testing utilities to fix rkt tests
2018-11-29 09:18:10 -05:00
Chris Baker
72ca56dabe testing: in MkAllocDir, do not update TaskConfig with All() from the task builder, just with Env() (because it pollutes environment variables with node attributes and fails the rkt tests) 2018-11-28 22:19:48 +00:00