Commit Graph

137 Commits

Author SHA1 Message Date
Michael Schurter
89e5971bc7 Merge pull request #2732 from hashicorp/b-persist-alloc-updates
Persist Alloc when EvalID changes
2017-07-03 14:46:43 -07:00
Michael Schurter
11863660a0 Destroy task group leader first
Before this commit all tasks in a task group were destroyed
concurrently. This meant logging sidecars might be stopped before the
leader task whose logs still need to be shipped.

This commit blocks on the leader shutting down before signalling to
followers to shutdown.
2017-07-03 13:56:56 -07:00
Michael Schurter
bb432213a6 Fix spelling & re-add immutable state struct 2017-06-23 13:01:39 -07:00
Michael Schurter
da4fb1f293 Rename immutable -> alloc
meh; naming is hard
2017-06-23 10:58:36 -07:00
Michael Schurter
38cb0f4af1 Persist Alloc when EvalID changes 2017-06-22 17:33:12 -07:00
Alex Dadgar
aa15ad51d1 Fix nil job on allocation
The way the copying was happening on the alloc_runner was by temporarily
setting the alloc.Job to nil, copying and then restoring it. This
created an issue in which when the alloc was shared (which it is in
server/client mode and between alloc_runner/task_runner) there were race
conditions that could create a panic.

Fixes https://github.com/hashicorp/nomad/issues/2605
2017-05-17 14:07:06 -04:00
Alex Dadgar
997390b04c Fix test 2017-05-09 11:35:48 -07:00
Alex Dadgar
e47be9f771 Merge branch 'master' into f-bolt-db 2017-05-09 11:11:55 -07:00
Alex Dadgar
3f1ccf7278 Respond to comments 2017-05-09 10:50:24 -07:00
Alex Dadgar
098b02d5bf Helpful comment 2017-05-03 11:27:33 -07:00
Alex Dadgar
5952dae3e0 Fix tests 2017-05-03 11:15:30 -07:00
Alex Dadgar
e22393aeb8 Restore state + upgrade path 2017-05-02 18:21:49 -07:00
Alex Dadgar
85a81f47de Revert "metrics"
This reverts commit 4d6a012c6f.
2017-05-02 09:28:11 -07:00
Alex Dadgar
4d6a012c6f metrics 2017-05-01 14:51:27 -07:00
Alex Dadgar
5aa6e18807 Use batching 2017-05-01 14:50:34 -07:00
Alex Dadgar
7614feddbd boltDB database for client state 2017-05-01 14:50:34 -07:00
Alex Dadgar
9def7e1a14 Don't deepcopy job when retrieving copy of Alloc
This PR removes deepcopying of the job attached to the allocation in the
alloc runner. This operation is called very often so removing reflect
from the code path and the potentially large number of mallocs need to
create a job reduced memory and cpu pressure.
2017-05-01 14:50:34 -07:00
Michael Schurter
90f5e232a5 Switch java/exec to use Exec in Executor 2017-04-21 16:25:49 -07:00
Michael Schurter
fe2b3e382d Restart tasks on upgrade with script checks and old executors 2017-04-21 16:25:49 -07:00
Michael Schurter
10cb924b2c Refactor Consul Syncer into new ServiceClient
Fixes #2478 #2474 #1995 #2294

The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.

The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.

Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.

Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
2017-04-19 12:42:47 -07:00
Alex Dadgar
3018ae07bb Sync allocation state before waiting for a destroy
This change ensures that the client syncs allocation state with the
servers before entering its wait loop for the allocation to be
destroyed.

Fixes https://github.com/hashicorp/nomad/issues/2563
2017-04-14 13:09:54 -07:00
Alex Dadgar
c934063713 FinishedAt only records when the task has actually started 2017-03-31 17:06:05 -07:00
Alex Dadgar
d212f6fe94 Track task start/finish time & improve logs errors
This PR adds tracking to when a task starts and finishes and the logs
API takes advantage of this and returns better errors when asking for
logs that do not exist.
2017-03-31 16:14:11 -07:00
Alex Dadgar
07973793b8 Address comment 2017-03-09 21:05:34 -08:00
Alex Dadgar
4fbe182372 Add metrics to show allocations on the client
This PR adds the following metrics to the client:
client.allocations.migrating
client.allocations.blocked
client.allocations.pending
client.allocations.running
client.allocations.terminal

Also adds some missing fields to the API version of the evaluation.
2017-03-09 12:37:41 -08:00
Alex Dadgar
07f7e19578 Fix vet script and fix vet problems
This PR fixes our vet script and fixes all the missed vet changes.

It also fixes pointers being printed in `nomad stop <job>` and `nomad
node-status <node>`.
2017-02-27 16:00:19 -08:00
Alex Dadgar
edbc84087c Add Leader support to client 2017-02-10 17:55:19 -08:00
Michael Schurter
9bfa8dc2bd Add COMPAT comment 2017-01-06 11:39:17 -08:00
Michael Schurter
e25274b775 Put a logger in AllocDir/TaskDir 2017-01-05 16:31:56 -08:00
Michael Schurter
3011df1c89 Fix upgrade path for #2132
AllocRunner's state dropped the Context struct which needs to be
converted to the new AllocDir+TaskDir structs in RestoreState.

TaskRunner added a TaskDirBuilt flag, but it's safe to just let that
default to `false` and rebuild all task dirs once on upgrade.
2017-01-05 16:31:55 -08:00
Michael Schurter
d87dcce722 Fail fast on taskdir errors 2017-01-05 16:31:55 -08:00
Michael Schurter
de7351b959 Move chroot building into TaskRunner
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
2017-01-05 16:31:49 -08:00
Bastiaan Bakker
bd0ab77a0b use snap.Alloc.TaskStates only after confirming snap.Alloc is not nil 2016-11-07 22:35:00 +01:00
Alex Dadgar
ad8b5bc141 Task state "dead" is terminal 2016-11-04 16:57:24 -07:00
Alex Dadgar
51831f92dd Merge pull request #1840 from hashicorp/f-kill-fail
Change how we mark tasks as failed and allow consul-template to fail tasks
2016-10-24 13:40:52 -07:00
Michael Schurter
4d3187bc7e Remove disk usage enforcement
Many thanks to @iverberk for the original PR (#1609), but we ended up
not wanting to ship this implementation with 0.5.

We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
2016-10-21 13:55:51 -07:00
Alex Dadgar
f130c8a894 Change how we mark tasks as failed and allow consul-template to fail tasks 2016-10-20 17:27:16 -07:00
Alex Dadgar
e34902ae8a Large refactor of task runner and Vault token rehandling 2016-10-18 11:24:20 -07:00
Diptanu Choudhury
c29861b418 Getting snapshot of allocation from remote node (#1741)
* Added the alloc dir move

* Moving allocdirs when starting allocations

* Added the migrate flag to ephemeral disk

* Stopping migration if the allocation doesn't need migration any more

* Added the GetAllocDir method

* refactored code

* Added a test for alloc runner

* Incorporated review comments
2016-10-03 09:59:57 -07:00
Alex Dadgar
d49dda45e3 Merge pull request #1713 from hashicorp/f-alloc-runner-vault
Vault integration in client
2016-09-20 16:15:55 -07:00
Alex Dadgar
5c3acf1b78 Fix comment 2016-09-17 11:31:17 -07:00
Alex Dadgar
038991b5e3 Handle recovery failure 2016-09-15 12:50:44 -07:00
Alex Dadgar
5532e911cb Fix token renewal 2016-09-15 11:20:51 -07:00
Alex Dadgar
c89fd0eb08 Clean up vault client 2016-09-14 18:10:56 -07:00
Alex Dadgar
bec6adb2ee Vault token threaded 2016-09-14 13:30:01 -07:00
Michael Schurter
78a4befa71 Revert "A nil context isn't an error"
This reverts commit fe9fe4c262.
2016-09-12 12:56:12 -07:00
Michael Schurter
fe9fe4c262 A nil context isn't an error 2016-09-02 16:24:53 -07:00
Michael Schurter
8cb85ed120 Don't serialize task states twice in state files 2016-09-02 16:07:06 -07:00
Michael Schurter
8d9468d4c3 Lock around saving state
Prevent interleaving state syncs as it could conceivably lead to
empty state files as per #1367
2016-09-02 16:07:06 -07:00
Michael Schurter
6f2b09f676 Add sanity check to SaveState
Also just reuse the task states snapshot taken by `Alloc()` instead of
doing a redundant copy.
2016-09-02 16:07:06 -07:00