Commit Graph

127 Commits

Author SHA1 Message Date
Mahmood Ali
b8f80e5124 Simplify proto conversion and handle swap
Convert all cpu and memory usage fields regardless of stated measured
fields, and handle swap fields
2019-03-30 15:18:28 -04:00
Mahmood Ali
484ccfafab deserialize total ticks 2019-03-30 07:14:57 -04:00
Mahmood Ali
a880f93a61 Always report TotalTicks when percent is measured
Fix a case where TotalTicks doesn't get serialized across executor grpc
calls.

Here, I opted to implicit add field, rather than explicitly mark it as a
measured field, because it's a derived field and to preserve 0.8
behavior where total ticks aren't explicitly marked as a measured field.
2019-03-29 22:34:28 -04:00
Mahmood Ali
7299e0015a Merge pull request #5428 from hashicorp/b-dropped-logs-on-task-restart
client/logmon: restart log collection correctly when a task is restarted
2019-03-21 14:02:08 -04:00
Nick Ethier
cc12c8443f logmon: fix logmon handling in driver test harness 2019-03-20 21:14:08 -04:00
Mahmood Ali
eb5ab38ae5 Regenerate Proto files (#5421)
Noticed that the protobuf files are out of sync with ones generated by 1.2.0 protoc go plugin.

The cause for these files seem to be related to release processes, e.g. [0.9.0-beta1 preperation](ecec3d38de (diff-da4da188ee496377d456025c2eab4e87)), and [0.9.0-beta3 preperation](b849d84f2f).

This restores the changes to that of the pinned protoc version and fails build if protobuf files are out of sync.  Sample failing Travis job is that of the first commit change: https://travis-ci.org/hashicorp/nomad/jobs/506285085
2019-03-14 10:56:27 -04:00
Michael Schurter
b849d84f2f Generate files for 0.9.0-beta3 release 2019-02-26 09:44:49 -08:00
Michael Schurter
c9fe5d26b3 plugins: squelch context Canceled error logs
As far as I can tell this is the most straightforward and resilient way
to skip error logging on context cancellation with grpc streams. You
cannot compare the error against context.Canceled directly as it is of
type `*status.statusError`. The next best solution I found was:

```go
resp, err := stream.Recv()
if code, ok := err.(interface{ Code() code.Code }); ok {
	if code.Code == code.Canceled {
		return
	}
}
```

However I think checking ctx.Err() directly makes the code much easier
to read and is resilient against grpc API changes.
2019-02-21 15:32:18 -08:00
Alex Dadgar
ecec3d38de Nomad 0.9.0-beta1 generated code 2019-01-30 10:49:44 -08:00
Nick Ethier
8aa451806b executor: fix bug and add tests for incorrect stats timestamp reporting 2019-01-28 21:57:45 -05:00
Mahmood Ali
b70442a325 drivers: pass logger through driver plugin client
This fixes a panic whenever driver plugin attempts to log a message.
2019-01-25 09:38:41 -05:00
Nick Ethier
f38612c3b3 plugins/drivers: change stats interval to duration type in proto 2019-01-24 22:19:18 -05:00
Michael Schurter
158c74887e goimports until make check is happy 2019-01-23 06:27:14 -08:00
Michael Schurter
0d61ff0fb9 move pluginutils -> helper/pluginutils
I wanted a different color bikeshed, so I get to paint it
2019-01-22 15:50:08 -08:00
Alex Dadgar
95297c608c goimports 2019-01-22 15:44:31 -08:00
Alex Dadgar
fe2fa21a7d gofmt 2019-01-22 15:43:34 -08:00
Alex Dadgar
b9f36134dc move catalog + grpcutils 2019-01-22 15:11:57 -08:00
Nick Ethier
f3ab65947e drivers: regen proto 2019-01-18 18:53:45 -05:00
Nick Ethier
994c66f7d7 drivers: use consts for task handle version 2019-01-18 18:31:01 -05:00
Nick Ethier
7c440aa437 drivers: fix test 2019-01-18 18:31:01 -05:00
Nick Ethier
2118d94b2d executor: add pre 0.9 client and wrapper 2019-01-18 18:30:58 -05:00
Mahmood Ali
b5c20aa50b Track Basic Memory Usage as reported by cgroups
Track current memory usage, `memory.usage_in_bytes`, in addition to
`memory.max_memory_usage_in_bytes` and friends.  This number is closer
what Docker reports.

Related to https://github.com/hashicorp/nomad/issues/5165 .
2019-01-14 18:47:52 -05:00
Nick Ethier
72a4685534 drivers: plumb grpc client logger 2019-01-12 12:18:23 -05:00
Nick Ethier
9904463da2 executor: fix failing stats related test 2019-01-12 12:18:23 -05:00
Nick Ethier
f6af1d4d04 docker: add test for stats collection 2019-01-12 12:18:22 -05:00
Nick Ethier
fbf9a4c772 executor: implement streaming stats API
plugins/driver: update driver interface to support streaming stats

client/tr: use streaming stats api

TODO:
 * how to handle errors and closed channel during stats streaming
 * prevent tight loop if Stats(ctx) returns an error

drivers: update drivers TaskStats RPC to handle streaming results

executor: better error handling in stats rpc

docker: better control and error handling of stats rpc

driver: allow stats to return a recoverable error
2019-01-12 12:18:22 -05:00
Mahmood Ali
d1fbd735f3 Merge pull request #5157 from hashicorp/r-drivers-no-cstructs
drivers: avoid referencing client/structs package
2019-01-09 13:06:46 -05:00
Mahmood Ali
855e3684e3 fixup! remove unused field 2019-01-08 12:58:12 -05:00
Mahmood Ali
b0a3f6b491 remove unused field 2019-01-08 12:19:44 -05:00
Mahmood Ali
34ee0ba6b9 Remove some dead code 2019-01-08 09:11:48 -05:00
Mahmood Ali
800a3522e3 drivers: re-export ResourceUsage structs
Re-export the ResourceUsage structs in drivers package to avoid drivers
directly depending on the internal client/structs package directly.

I attempted moving the structs to drivers, but that caused some import
cycles that was a bit hard to disentagle.  Alternatively, I added an
alias here that's sufficient for our purposes of avoiding external
drivers depend on internal packages, while allowing us to restructure
packages in future without breaking source compatibility.
2019-01-08 09:11:47 -05:00
Mahmood Ali
c0162fab35 move cstructs.DeviceNetwork to drivers pkg 2019-01-08 09:11:47 -05:00
Mahmood Ali
694e3010c2 use drivers.FSIsolation 2019-01-08 09:11:47 -05:00
Danielle Tomlinson
994057abf4 drivers: Add internal interface for Shutdown
This allows us to correctly terminate internal state during runs of the
nomad test suite, e.g closing eventer contexts correctly.
2019-01-08 13:48:49 +01:00
Alex Dadgar
4e7da3ea4e regenerate protos 2019-01-07 14:49:40 -08:00
Alex Dadgar
437f03d877 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar
ffadab1b20 remove nil logger 2019-01-07 14:48:01 -08:00
Preetha Appan
26594aa31e Standardize driver health description messages for all drivers 2019-01-06 22:06:38 -06:00
Danielle Tomlinson
749eab7257 chore: Fix environement->environment typo 2019-01-03 13:31:26 +01:00
Danielle Tomlinson
26b5ad4532 Merge pull request #5041 from hashicorp/dani/b-driver-healt
drivers: Cleanup root user fingerprinting
2019-01-03 13:16:28 +01:00
Danielle Tomlinson
4798dacafd plugins: Add consistent message for requires root 2018-12-20 12:54:01 +01:00
Alex Dadgar
99df4c98c7 Store device envs separately and pass to drivers 2018-12-19 14:23:09 -08:00
Alex Dadgar
9a017adfc9 proto 2018-12-19 13:54:19 -08:00
Nick Ethier
6951ca487d drivermanager: use allocID and task name to route task events 2018-12-18 23:01:51 -05:00
Alex Dadgar
ed4f8eac6e Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Alex Dadgar
da6925bfc1 utilities 2018-12-18 15:48:52 -08:00
Alex Dadgar
e1cf3ac69e protos 2018-12-18 15:48:52 -08:00
Nick Ethier
81ba18d74a executor: encode mounts and devices correctly when using grpc 2018-12-15 00:08:23 -05:00
Nick Ethier
8a344412e8 Merge branch 'master' into f-grpc-executor
* master: (71 commits)
  Fix output of 'nomad deployment fail' with no arg
  Always create a running allocation when testing task state
  tests: ensure exec tests pass valid task resources (#4992)
  some changes for more idiomatic code
  fix iops related tests
  fixed bug in loop delay
  gofmt
  improved code for readability
  client: updateAlloc release lock after read
  fixup! device attributes in `nomad node status -verbose`
  drivers/exec: support device binds and mounts
  fix iops bug and increase test matrix coverage
  tests: tag image explicitly
  changelog
  ci: install lxc-templates explicitly
  tests: skip checking rdma cgroup
  ci: use Ubuntu 16.04 (Xenial) in TravisCI
  client: update driver info on new fingerprint
  drivers/docker: enforce volumes.enabled (#4983)
  client: Style: use fluent style for building loggers
  ...
2018-12-13 14:41:09 -05:00
Nick Ethier
a6cb63a964 executor: don't drop errors when configuring libcontainer cfg, add nil check on resources 2018-12-07 14:03:42 -05:00