Commit Graph

3843 Commits

Author SHA1 Message Date
Mahmood Ali
7c1fe3eae5 logmon: log stat error to help debugging 2019-08-26 10:10:20 -04:00
Mahmood Ali
cc3da4a441 logmon: revert workaround for Windows go1.11 bug
Revert e0126123ab now that we are running
with Golang 1.12, and https://github.com/golang/go/issues/29119 is no
longer relevant.
2019-08-24 08:19:44 -04:00
Mahmood Ali
b4a80a7eea Merge pull request #6201 from hashicorp/b-device-stats-interval
initialize device manager stats interval
2019-08-24 08:16:03 -04:00
Mahmood Ali
e87d9cc8a6 Merge pull request #6146 from hashicorp/b-config-template-copy
clientConfig.Copy() to copy template config too
2019-08-23 19:00:57 -04:00
Mahmood Ali
e8ebde4ca2 clientConfig.Copy() to copy template config too 2019-08-23 18:43:22 -04:00
Lang Martin
5fc06cd65f taskrunner getter set Umask for go-getter, setuid test 2019-08-23 15:59:03 -04:00
Mahmood Ali
01983ae59b initialize device manager stats interval
Fixes a bug where we cpu is pigged at 100% due to collecting devices
statistics.  The passed stats interval was ignored, and the default zero
value causes a very tight loop of stats collection.

FWIW, in my testing, it took 2.5-3ms to collect nvidia GPU stats, on a
`g2.2xlarge` ec2 instance.

The stats interval defaults to 1 second and is user configurable.  I
believe this is too frequent as a default, and I may advocate for
reducing it to a value closer to 5s or 10s, but keeping it as is for
now.

Fixes https://github.com/hashicorp/nomad/issues/6057 .
2019-08-23 14:58:34 -04:00
Jerome Gravel-Niquet
25e38c8257 Consul service meta (#6193)
* adds meta object to service in job spec, sends it to consul

* adds tests for service meta

* fix tests

* adds docs

* better hashing for service meta, use helper for copying meta when registering service

* tried to be DRY, but looks like it would be more work to use the
helper function
2019-08-23 12:49:02 -04:00
Nick Ethier
974ff0392c ar: fix bridge networking port mapping when port.To is unset (#6190) 2019-08-22 21:53:52 -04:00
Michael Schurter
43d89f864e connect: task hook for bootstrapping envoy sidecar
Fixes #6041

Unlike all other Consul operations, boostrapping requires Consul be
available. This PR tries Consul 3 times with a backoff to account for
the group services being asynchronously registered with Consul.
2019-08-22 08:15:32 -07:00
Michael Schurter
eeacb87f3b connect: register group services with Consul
Fixes #6042

Add new task group service hook for registering group services like
Connect-enabled services.

Does not yet support checks.
2019-08-20 12:25:10 -07:00
Nick Ethier
aaba483787 Builtin Admission Controller Framework (#6116)
* nomad: add admission controller framework

* nomad: add admission controller framework and Consul Connect hooks

* run admission controllers before checking permissions

* client: add default node meta for connect configurables

* nomad: remove validateJob func since it has been moved to admission controller

* nomad: use new TaskKind type

* client: use consts for connect sidecar image and log level

* Apply suggestions from code review

Co-Authored-By: Michael Schurter <mschurter@hashicorp.com>

* nomad: add job register test with connect sidecar

* Update nomad/job_endpoint_hooks.go

Co-Authored-By: Michael Schurter <mschurter@hashicorp.com>
2019-08-15 11:22:37 -04:00
Tim Gross
ffb83e1ef1 client/template: configuration for function blacklist and sandboxing
When rendering a task template, the `plugin` function is no longer
permitted by default and will raise an error. An operator can opt-in
to permitting this function with the new `template.function_blacklist`
field in the client configuration.

When rendering a task template, path parameters for the `file`
function will be treated as relative to the task directory by
default. Relative paths or symlinks that point outside the task
directory will raise an error. An operator can opt-out of this
protection with the new `template.disable_file_sandbox` field in the
client configuration.
2019-08-12 16:34:48 -04:00
Danielle Lancashire
c486143ced Copy documentation to api/tasks 2019-08-12 16:22:27 +02:00
Danielle Lancashire
7b7be83aef HostVolumeConfig: Source -> Path 2019-08-12 15:39:08 +02:00
Danielle Lancashire
af5d42c058 structs: Unify Volume and VolumeRequest 2019-08-12 15:39:08 +02:00
Danielle Lancashire
c3c003dbd6 client: Add volume_hook for mounting volumes 2019-08-12 15:39:08 +02:00
Danielle Lancashire
86b4296f9d client: Add parsing and registration of HostVolume configuration 2019-08-12 15:39:08 +02:00
Nick Ethier
144fb1bfee Revert "client: add autofetch for CNI plugins"
This reverts commit 0bd157cc3b.
2019-08-08 15:10:19 -04:00
Nick Ethier
4b814be995 Revert "client: remove debugging lines"
This reverts commit 54ce4d1f7e.
2019-08-08 14:52:52 -04:00
Mahmood Ali
b6ae5e9d62 Render consul templates using task env only (#6055)
When rendering a task consul template, ensure that only task environment
variables are used.

Currently, `consul-template` always falls back to host process
environment variables when key isn't a task env var[1].  Thus, we add
an empty entry for each host process env-var not found in task env-vars.

[1] bfa5d0e133/template/funcs.go (L61-L75)
2019-08-05 16:30:47 -04:00
Mahmood Ali
2e0d67cbbb Merge pull request #6065 from hashicorp/b-nil-driver-exec
Check if driver handle is nil before execing
2019-08-02 09:48:28 -05:00
Mahmood Ali
488cd7e24e Check if driver handle is nil before execing
Defend against tr.getDriverHandle being nil.  Exec handler checks if
task is running, but it may be stopped between check and driver handler
fetching.
2019-08-02 10:07:41 +08:00
Nick Ethier
0b8fc5d018 client/cni: updated comments and simplified logic to auto download plugins 2019-07-31 01:04:10 -04:00
Nick Ethier
1072084ff3 Apply suggestions from code review
Co-Authored-By: Mahmood Ali <mahmood@hashicorp.com>
2019-07-31 01:04:10 -04:00
Nick Ethier
54ce4d1f7e client: remove debugging lines 2019-07-31 01:04:09 -04:00
Nick Ethier
0bd157cc3b client: add autofetch for CNI plugins 2019-07-31 01:04:09 -04:00
Nick Ethier
4dd5cd103d remove unused file 2019-07-31 01:04:09 -04:00
Nick Ethier
e910fdbb32 fix failing tests 2019-07-31 01:04:07 -04:00
Nick Ethier
dc08ec8783 ar: plumb client config for networking into the network hook 2019-07-31 01:04:06 -04:00
Nick Ethier
e15005bdcb networking: Add new bridge networking mode implementation 2019-07-31 01:04:06 -04:00
Michael Schurter
eb2a2cd76e connect: add group.service stanza support 2019-07-31 01:04:05 -04:00
Nick Ethier
9fa47daf5c ar: fix lint errors 2019-07-31 01:03:19 -04:00
Nick Ethier
56d5fe704a ar: rearrange network hook to support building on windows 2019-07-31 01:03:19 -04:00
Nick Ethier
508ff3cf2f ar: fix test that failed due to error renaming 2019-07-31 01:03:19 -04:00
Nick Ethier
da3978b377 plugins/driver: make DriverNetworkManager interface optional 2019-07-31 01:03:19 -04:00
Nick Ethier
35de444e9b ar: plumb error handling into alloc runner hook initialization 2019-07-31 01:03:18 -04:00
Nick Ethier
16343ae23a ar: add tests for network hook 2019-07-31 01:03:18 -04:00
Nick Ethier
c742f8b580 ar: cleanup lint errors 2019-07-31 01:03:18 -04:00
Nick Ethier
c39e8dca6e ar: move linux specific code to it's own file and add tests 2019-07-31 01:03:18 -04:00
Nick Ethier
4a8a96fa1a ar: initial driver based network management 2019-07-31 01:03:17 -04:00
Nick Ethier
e20fa7ccc1 Add network lifecycle management
Adds a new Prerun and Postrun hooks to manage set up of network namespaces
on linux. Work still needs to be done to make the code platform agnostic and
support Docker style network initalization.
2019-07-31 01:03:17 -04:00
Preetha Appan
ebd7002683 remove generated code and change version to 0.10.0 2019-07-30 15:56:05 -05:00
Nomad Release bot
a81aa846a4 Generate files for 0.9.4 release 2019-07-30 19:05:18 +00:00
Preetha Appan
31d674bf39 remove generated code 2019-07-23 12:07:49 -05:00
Nomad Release bot
4999923574 Generate files for 0.9.4-rc1 release 2019-07-22 21:42:36 +00:00
Michael Schurter
1a90b2d082 logmon: fix comment formattinglogmon: fix comment formattinglogmon: fix
comment formattinglogmon: fix comment formattinglogmon: fix comment
formatting
2019-07-22 13:05:01 -07:00
Michael Schurter
12c6ceb27b logmon: ensure errors are still handled properly
...and add a comment to switch back to the old error handling once we
switch to Go 1.12.
2019-07-22 12:49:48 -07:00
Danielle Lancashire
e0126123ab logmon: Workaround golang/go#29119
There's a bug in go1.11 that causes some io operations on windows to
return incorrect errors for some cases when Stat-ing files. To avoid
upgrading to go1.12 in a point release, here we loosen up the cases
where we will attempt to create fifos, and add some logging of
underlying stat errors to help with debugging.
2019-07-22 18:28:12 +02:00
Jasmine Dahilig
e31db578e0 add formatting for hcl parsing error messages (#5972) 2019-07-19 10:04:39 -07:00