Commit Graph

14763 Commits

Author SHA1 Message Date
Mahmood Ali
bfb4f0ca2d agent: add websocket handler for nomad exec
This adds a websocket endpoint for handling `nomad exec`.

The endpoint is a websocket interface, as we require a bi-directional
streaming (to handle both input and output), which is not very appropriate for
plain HTTP 1.0. Using websocket makes implementing the web ui a bit simpler. I
considered using golang http hijack capability to treat http request as a plain
connection, but the web interface would be too complicated potentially.

Furthermore, the API endpoint operates against the raw core nomad exec streaming
datastructures, defined in protobuf, with json serializer.  Our APIs use json
interfaces in general, and protobuf generates json friendly golang structs.
Reusing the structs here simplify interface and reduce conversion overhead.
2019-05-09 16:49:08 -04:00
Mahmood Ali
a77a3ba9b0 server: server forwarding logic for nomad exec endpoint 2019-05-09 16:49:08 -04:00
Mahmood Ali
979a6a1778 implement client endpoint of nomad exec
Add a client streaming RPC endpoint for processing nomad exec tasks, by invoking
the relevant task handler for execution.
2019-05-09 16:49:08 -04:00
Mahmood Ali
cb4ad3fb45 drivers/mock: implement nomad exec interface 2019-05-09 16:49:08 -04:00
Mahmood Ali
94ed649489 implemment streaming exec handling in driver grpc handlers
Also add a helper that converts the adapts the high level interface to the
low-level interface of nomad exec interfaces.
2019-05-09 16:49:08 -04:00
Mahmood Ali
6d711d054b add nomad streaming exec core data structures and interfaces
In this commit, we add two driver interfaces for supporting `nomad exec`
invocation:

* A high level `ExecTaskStreamingDriver`, that operates on io reader/writers.
  Drivers should prefer using this interface
* A low level `ExecTaskStreamingRawDriver` that operates on the raw stream of
  input structs; useful when a driver delegates handling to driver backend (e.g.
  across RPC/grpc).

The interfaces are optional for a driver, as `nomad exec` support is opt-in.
Existing drivers continue to compile without exec support, until their
maintainer add such support.

Furthermore, we create protobuf structures to represent exec stream entities:
`ExecTaskStreamingRequest` and `ExecTaskStreamingResponse`.  We aim to reuse the
protobuf generated code as much as possible, without translation to avoid
conversion overhead.

`ExecTaskStream` abstract fetching and sending stream entities.  It's influenced
by the grpc bi-directional stream interface, to avoid needing any adapter.  I
considered using channels, but the asynchronisity and concurrency makes buffer
reuse too complicated, which would put more pressure on GC and slows exec operation.
2019-04-30 14:02:29 -04:00
Mahmood Ali
4f7bd68f5c Add ACL capabilities for nomad exec
This adds `alloc-exec` capability to allow operator to execute command into a
running task.  Furthermore, it adds `alloc-node-exec` capability, required when
the alloc task is raw_exec or a driver with no FSIsolation.
2019-04-30 14:02:16 -04:00
Mahmood Ali
e79ce1f9d0 drivers/mock: extract command related operations
Extract command parsing and execution mocking into a separate struct.  Also,
allow mocking of different fs_isolation for testing.
2019-04-30 14:02:16 -04:00
Mahmood Ali
75349d647e test helper for registering jobs with acl
Test helper that allows registration of jobs when ACL is activated.
2019-04-30 10:23:56 -04:00
Mahmood Ali
43cd7a71a6 aux: helper method that returns token as well as ACL policy
This helper returns the token as well as the ACL policy, to be used in a later
commit for logging the token info associated with nomad exec invocation.
2019-04-30 10:23:56 -04:00
Charlie Voiselle
15d40ac22f Merge pull request #5630 from jweissig/patch-5
Fixed typo
2019-04-30 08:59:16 -04:00
Mahmood Ali
f94fd532ed update 0.9.1 changelog 2019-04-30 07:48:00 -04:00
Danielle
90036dea15 Merge pull request #5614 from hashicorp/dani/we-released-a-hotfix
Update CHANGELOG and version.go to reflect 0.9.1/0.9.2 split
2019-04-30 10:35:22 +02:00
Danielle Lancashire
8179d4aa21 Update master to reflect 0.9.1/0.9.2 split 2019-04-30 10:33:41 +02:00
Justin Weissig
9f7703c56e Fixed typo
Fixed typo: "dots wil be"/"dots will be".
2019-04-30 00:04:17 -07:00
Yishan Lin
51a4fee2f7 Merge pull request #5618 from hashicorp/yishan/revised-readme
yishan/revised readme
2019-04-29 15:46:20 -07:00
Yishan Lin
b78b0f98fc Addressed Dani's comments. 2019-04-29 10:22:27 -07:00
Preetha Appan
6ca6b85ba8 Update 0.9.1 links in website 2019-04-29 11:46:50 -05:00
Danielle
7e2f78a7e9 Merge pull request #5623 from jweissig/patch-4
Fixed Typo
2019-04-29 10:25:45 +02:00
Justin Weissig
46f3376134 Fixed Typo
Fixed typo: scheduilng/scheduling.
2019-04-29 00:41:53 -07:00
Mahmood Ali
634592eb15 Merge pull request #5622 from hashicorp/b-divest-api-20190427
divest /api from rest of nomad internal packages
2019-04-28 16:24:08 -04:00
Mahmood Ali
1a54a0b839 divest /api from nomad/structs
The API package needs to be independent from rest of nomad packages, to
avoid leaking internal packages and dependencies (e.g. raft, ugorji,
etc)
2019-04-28 13:32:26 -04:00
Mahmood Ali
5aaae70a99 check that /api doesn't import internal nomad pkgs 2019-04-28 13:32:26 -04:00
Lang Martin
5ae04a4e76 CHANGELOG.md add fix line for #2619 2019-04-26 16:33:17 -04:00
Yishan Lin
317cb174e3 Update README.md 2019-04-26 11:48:45 -07:00
Yishan Lin
0c51383d54 Update README.md 2019-04-26 11:48:13 -07:00
Yishan Lin
80e3cf73cc Update README.md 2019-04-26 11:46:30 -07:00
Yishan Lin
9e0b3295c7 Reordered guide lists + formatting with users/content. 2019-04-26 11:45:43 -07:00
Yishan Lin
c7b53642df updated hyperlinks. 2019-04-26 11:43:16 -07:00
Yishan Lin
1ff9798daa Update README.md 2019-04-26 11:28:09 -07:00
Lang Martin
33f550fb52 Merge pull request #5553 from hashicorp/b-fingerprinter-manual-config
client fingerprinter doesn't overwrite manual configuration
2019-04-26 12:55:34 -04:00
Mahmood Ali
4200d7116e Merge pull request #5370 from hashicorp/f-windows-volume-binds
drivers/docker: Support volumes field in Windows
2019-04-26 10:12:42 -04:00
Danielle
d38789cfc2 Merge pull request #5612 from hashicorp/dani/alloc-signal-docs
Docs for `nomad alloc signal`
2019-04-26 14:21:23 +02:00
Danielle
91fa55f4cc Merge pull request #5515 from hashicorp/dani/f-alloc-signal
allocs: Add nomad alloc signal command
2019-04-26 14:21:05 +02:00
Danielle Lancashire
7f102bcea8 alloc_signal: Add autcompletion and cmd tests 2019-04-26 12:47:53 +02:00
Mahmood Ali
1497b8ee00 Merge pull request #5616 from hashicorp/b-retry-logmon-start-errs-2
logmon: recover from shutting down call locally
2019-04-25 21:42:09 -04:00
Mahmood Ali
a321901ad8 retry grpc unavailable errors even if not shutting down 2019-04-25 18:39:17 -04:00
Mahmood Ali
658a734912 try checking process status 2019-04-25 18:16:13 -04:00
Mahmood Ali
1f1551a4ae add logging about attempts 2019-04-25 18:09:36 -04:00
Mahmood Ali
ba373fee2a try sleeping for stop signal to take effect 2019-04-25 17:16:29 -04:00
Mahmood Ali
978fc65a2b add a test that simulates logmon dying during Start() call 2019-04-25 16:41:17 -04:00
Mahmood Ali
b21849cb02 logmon: retry starting logmon if it exits
Retry if we detect shutting down during Start() api call is started,
locally.
2019-04-25 15:10:16 -04:00
Mahmood Ali
c23d673b7e logmon client to handle grpc closing errors 2019-04-25 14:32:24 -04:00
Preetha
81c7c467a7 Update CHANGELOG.md 2019-04-25 12:51:07 -05:00
Mahmood Ali
fce6564ce4 driver/docker: Support volumes field in Windows
Support Docker `volumes` field in Windows.  Previously, volumes parser
assumed some Unix-ism (e.g. didn't expect `:` in mount paths).
Here, we use the Docker parser to identify host and container paths.

Docker parsers use different validation logic from our previous unix
implementation: Docker parser accepts single path as a volume entry
(parsing it as a container path with auto-created volume) and enforces
additional checks (e.g. validity of mode).  Thereforce, I opted to use
Docker parser only for Windows, and keep Nomad's linux parser to
preserve current behavior.
2019-04-25 09:02:44 -04:00
Mahmood Ali
e8a4610017 vendor docker/docker volume utils 2019-04-25 08:55:21 -04:00
Danielle Lancashire
05ef73c376 docs: Fix copy/paste error in alloc-restart docs 2019-04-25 14:40:09 +02:00
Danielle Lancashire
e97dfce7e1 docs: Add nomad alloc signal docs 2019-04-25 14:39:49 +02:00
Mahmood Ali
2764fc8c3c Merge pull request #5609 from hashicorp/b-docker-tty-logs
driver/docker: collect tty container logs
2019-04-25 08:28:32 -04:00
Danielle Lancashire
023d0dff31 allocs: Add nomad alloc signal command
This command will be used to send a signal to either a single task within an
allocation, or all of the tasks if <task-name> is omitted. If the sent signal
terminates the allocation, it will be treated as if the allocation has crashed,
rather than as if it was operator-terminated.

Signal validation is currently handled by the driver itself and nomad
does not attempt to restrict or validate them.
2019-04-25 12:43:32 +02:00