Commit Graph

193 Commits

Author SHA1 Message Date
Danielle Tomlinson
04577f7e15 executor: Fix stdout stderr copy/paste 2018-11-12 22:08:04 -08:00
Preetha Appan
4231dc4729 Stat path to binary to handle raw exec driver interpolated binary path 2018-10-26 17:24:05 -05:00
Nick Ethier
7f5046849e drivers/exec: add exec implementation 2018-10-16 22:45:28 -04:00
Nick Ethier
c9f0d2e0b4 driver/raw_exec: port existing raw_exec tests and add some testing utilities 2018-10-16 16:53:31 -07:00
Nick Ethier
650ac5a83e driver/raw_exec: more tests and bug fixes
added wrapper struct for plugin.ReattachConfig to better handle serialization
2018-10-16 16:53:31 -07:00
Nick Ethier
e2bf0a388e clientv2: base driver plugin (#4671)
Driver plugin framework to facilitate development of driver plugins.

Implementing plugins only need to implement the DriverPlugin interface.
The framework proxies this interface to the go-plugin GRPC interface generated
from the driver.proto spec.

A testing harness is provided to allow implementing drivers to test the full
lifecycle of the driver plugin. An example use:

func TestMyDriver(t *testing.T) {
    harness := NewDriverHarness(t, &MyDiverPlugin{})
    // The harness implements the DriverPlugin interface and can be used as such
    taskHandle, err := harness.StartTask(...)
}
2018-10-16 16:53:31 -07:00
Nick Ethier
5b14d24bf4 executor v2 (#4656)
* client/executor: refactor client to remove interpolation

* executor: POC libcontainer based executor

* vendor: use hashicorp libcontainer fork

* vendor: add libcontainer/nsenter dep

* executor: updated executor interface to simplify operations

* executor: implement logging pipe

* logmon: new logmon plugin to manage task logs

* driver/executor: use logmon for log management

* executor: fix tests and windows build

* executor: fix logging key names

* executor: fix test failures

* executor: add config field to toggle between using libcontainer and standard executors

* logmon: use discover utility to discover nomad executable

* executor: only call libcontainer-shim on main in linux

* logmon: use seperate path configs for stdout/stderr fifos

* executor: windows fixes

* executor: created reusable pid stats collection utility that can be used in an executor

* executor: update fifo.Open calls

* executor: fix build

* remove executor from docker driver

* executor: Shutdown func to kill and cleanup executor and its children

* executor: move linux specific universal executor funcs to seperate file

* move logmon initialization to a task runner hook

* client: doc fixes and renaming from code review


* taskrunner: use shared config struct for logmon fifo fields

* taskrunner: logmon only needs to be started once per task
2018-10-16 16:53:31 -07:00
Michael Schurter
5b09d23813 executor: fix Windows blocking on pipe close
Sending the Ctrl-Break signal to PowerShell <6 causes it to drop into
debug mode. Closing its output pipe at that point will block
indefinitely and prevent the process from being killed by Nomad.

See the upstream powershell issue for details:
https://github.com/PowerShell/PowerShell/issues/4254
2018-06-08 14:48:05 -07:00
Alex Dadgar
a4e11c8134 Comments 2018-05-31 18:05:03 -07:00
Alex Dadgar
207fece24e Use more appropriate name than cgroup 2018-05-31 18:05:03 -07:00
Alex Dadgar
1adf3f9047 Use cgroup when possible 2018-05-31 18:05:03 -07:00
Alex Dadgar
60ef13c521 spelling 2018-05-31 15:29:55 -07:00
Alex Dadgar
4150296b46 Force close stdout/stderr after grace
This commit changes the force closing of the stdout/stderr file
descriptor from closing immediately to being closed after a grace
period. This allows the created process to close its own file and allows
copying of the data.
2018-05-31 15:21:36 -07:00
Alex Dadgar
52bed806fc cleanup 2018-05-24 16:25:20 -07:00
Alex Dadgar
7ece3a7d77 Force closing of pipe to child process 2018-05-24 16:03:48 -07:00
Michael Schurter
a4bf901559 run goimports 2018-04-19 11:16:28 -07:00
Fabien Ninoles
d6cc8895ca Update based on PR request. 2018-04-17 13:43:04 -04:00
Fabien Ninoles
7023b63ffa - Clean up for windows compilation.
- Set CREATE_NEW_PROCESS_GROUP for Windows subprocess.
- Ensure we only kill actual process that need to.
2018-04-14 13:58:42 -04:00
Michael Schurter
1e67780f1e Merge pull request #3572 from emate/master
Create new process group on process startup.
2018-04-13 11:56:38 -07:00
Michael Schurter
2df0640e47 executor: increase level for helpful log lines
Should help with debugging issues like #3971
2018-03-21 11:53:58 -07:00
Marcin Matlaszek
9b5a5922e2 Make raw_exec processes cleanup function more precise. 2018-03-20 13:40:21 +01:00
Marcin Matlaszek
963c10e584 Fix errors when trying to kill whole process group. 2018-03-20 13:40:21 +01:00
Marcin Matlaszek
76a8978c3e Make starting & cleaning process group Windows compatible. 2018-03-20 13:40:21 +01:00
Marcin Matlaszek
708d74cb7a Create new process group on process startup.
Clean up by sending SIGKILL to the whole process group.
2018-03-20 13:40:21 +01:00
Josh Soref
31651d124d spelling: remnants 2018-03-11 18:41:26 +00:00
Josh Soref
bd2912978b spelling: isolation 2018-03-11 18:19:02 +00:00
Chelsea Holland Komlo
f09e3ec798 set default kill signal on executor shutdown 2017-12-07 11:40:15 -05:00
Chelsea Holland Komlo
927c0a4d88 change location of default kill signal 2017-12-06 17:48:25 -05:00
Chelsea Holland Komlo
9e49836d4f move kill_signal to task level, extend to docker 2017-12-06 14:36:39 -05:00
Chelsea Holland Komlo
079e6b5fb1 allow controlling the stop signal in exec/raw_exec 2017-12-06 11:28:45 -05:00
Adam Stankiewicz
79eccb2b1e Remove AllocID from ExecutorContext 2017-10-13 17:07:49 +02:00
Alex Dadgar
a9e3a41407 Enable more linters 2017-09-26 15:26:33 -07:00
Michael Schurter
52ffc01972 Log PID when sending signals 2017-06-12 11:11:36 -07:00
Michael Schurter
ace00980a1 Refactor TaskEnvironment into Builder and TaskEnv 2017-05-23 13:53:33 -07:00
Michael Schurter
426c696404 Reuse ExecScript implementation 2017-05-04 16:21:40 -07:00
Michael Schurter
90f5e232a5 Switch java/exec to use Exec in Executor 2017-04-21 16:25:49 -07:00
Michael Schurter
10cb924b2c Refactor Consul Syncer into new ServiceClient
Fixes #2478 #2474 #1995 #2294

The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.

The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.

Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.

Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
2017-04-19 12:42:47 -07:00
Alex Dadgar
36dc330737 Various fixes
This PR:
* Uses Go 1.8 executable lookup
* Stores any err message from stats init method
* Allows overriding of Cpu Compute for hosts where it can't be detected
2017-03-14 12:56:31 -07:00
Alex Dadgar
07f7e19578 Fix vet script and fix vet problems
This PR fixes our vet script and fixes all the missed vet changes.

It also fixes pointers being printed in `nomad stop <job>` and `nomad
node-status <node>`.
2017-02-27 16:00:19 -08:00
Diptanu Choudhury
84137b02c5 Filter executor log messages 2017-01-12 11:54:19 -08:00
Michael Schurter
12915c30bf Remove debug logging 2017-01-05 16:31:56 -08:00
Michael Schurter
de7351b959 Move chroot building into TaskRunner
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
2017-01-05 16:31:49 -08:00
Alex Dadgar
3bf19b555e Interpolate all check related variables 2016-11-08 14:43:46 -08:00
Diptanu Choudhury
6acfa45f2a Making the executor lock the logger 2016-10-28 10:57:35 -07:00
Alex Dadgar
a186f7a26d Merge pull request #1806 from hashicorp/f-docker4mac-fixes
A couple fixes to make Docker For Mac work
2016-10-27 09:29:40 -07:00
Alex Dadgar
00ce1e50f5 Fix panic 2016-10-25 17:27:13 -07:00
Alex Dadgar
917c7e5085 Merge pull request #1801 from hashicorp/f-signals
Consul-template signal change mode
2016-10-18 11:23:47 -07:00
Alex Dadgar
358c43333d Send Executor Ctx separately 2016-10-12 11:35:29 -07:00
Alex Dadgar
0b65a5f95d Executor + Java/Raw Exec/Exec 2016-10-10 11:47:04 -07:00
Alex Dadgar
42c4346dcf Constant time space tracking of pids 2016-09-27 16:57:26 -07:00