Commit Graph

76 Commits

Author SHA1 Message Date
Alex Dadgar
3c689ba97f Client Stat/List impl 2018-02-15 13:59:02 -08:00
Alex Dadgar
5e7a1a44a2 Logs over RPC w/ lots to touch up 2018-02-15 13:59:01 -08:00
Michael Schurter
50b335f3f0 Add comment and normalize err check ordering
as per PR comments
2017-11-29 17:26:11 -08:00
Michael Schurter
e3256ec4ee Check for error file when receiving snapshots 2017-11-29 17:26:11 -08:00
Michael Schurter
a05862dbdf Destroy partially migrated alloc dirs
Test that snapshot errors don't return a valid tar currently fails.
2017-11-29 17:26:11 -08:00
Michael Schurter
4a49375988 Handle errors during snapshotting
If an alloc dir is being GC'd (removed) during snapshotting the walk
func will be passed an error. Previously we didn't check for an error so
a panic would occur when we'd try to use a nil `fileInfo`.
2017-11-29 17:26:11 -08:00
Michael Schurter
8983bc07cc spelling 2017-08-14 16:55:59 -07:00
Michael Schurter
8c1811911e switch from alloc blocker to new interface
interface has 3 implementations:

1. local for blocking and moving data locally
2. remote for blocking and moving data from another node
3. noop for allocs that don't need to block
2017-08-11 16:21:35 -07:00
Michael Schurter
12d9e91f65 Ensure allocDir is never nil and persisted safely
Fixes #2834
2017-07-17 15:46:54 -07:00
Mark Mickan
9e984f429c Include symlinks in snapshots when migrating disks
Fixes #2685
2017-06-04 00:36:18 +09:30
Michael Schurter
10cb924b2c Refactor Consul Syncer into new ServiceClient
Fixes #2478 #2474 #1995 #2294

The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.

The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.

Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.

Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
2017-04-19 12:42:47 -07:00
Michael Schurter
17471bf7c0 Set ownership on directories in chroot
Also support getOwner on all Unixes as they all have `Stat_t.{U,G}id`
2017-04-17 12:41:33 -07:00
Michael Schurter
2fe3de16be Chown files when copying into chroot
Fixes #2552

Not needed when hardlinking. Only adds Linux support but other OS's may
be easy.
2017-04-17 11:03:15 -07:00
Alex Dadgar
274c855f00 Add sticky bit to temp directory
Fixes an issue where the Ruby runtime expects the sticky bit to be set
on the temp directory. The sticky bit is commonly set on the temp
directory since it is usually shared by many users. This change brings
ours in line with that assumption.
2017-04-04 10:48:29 -07:00
Michael Schurter
ad4559d019 Safely ensure {dev,proc,alloc} are mounted
If they're unmounted by a reboot they'll be properly remounted.
2017-03-02 13:21:34 -08:00
Michael Schurter
da3e34710c Fix allocdir Move test and make code more defensive
A change in the behavior of `os.Rename` in Go 1.8 brought to light a
difference in the logic between `{Alloc,Task}Runner` and this test:

AllocRunner builds the alloc dir, moves dirs if necessary, and then lets
TaskRunner call TaskDir.Build().

This test called `TaskDir.Build` *before* `AllocDir.Move`, so in Go 1.8
it failed to `os.Rename over` the empty {data,local} dirs.

I updated the test to behave like the real code, but I defensively added
`os.Remove` calls as a subtle change in call order shouldn't break this
code. `os.Remove` won't remove a non-empty directory, so it's still
safe.
2017-02-21 17:22:10 -08:00
Michael Schurter
e25274b775 Put a logger in AllocDir/TaskDir 2017-01-05 16:31:56 -08:00
Michael Schurter
de7351b959 Move chroot building into TaskRunner
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
2017-01-05 16:31:49 -08:00
Alex Dadgar
732cce5d4a ensure file doesn't escape 2016-12-18 15:48:30 -08:00
Diptanu Choudhury
0beb8c0856 Fixed comments 2016-11-08 12:55:15 -08:00
Diptanu Choudhury
68aedd3f8f Fixed permission issues on client 2016-11-08 10:57:29 -08:00
Alex Dadgar
d4aa3fb831 Disallow fs to read secret directory 2016-10-24 11:14:05 -07:00
Michael Schurter
4d3187bc7e Remove disk usage enforcement
Many thanks to @iverberk for the original PR (#1609), but we ended up
not wanting to ship this implementation with 0.5.

We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
2016-10-21 13:55:51 -07:00
Ben Barnard
ce94317d00 Replace "the the" with "the" in documentation and comments 2016-10-11 15:31:40 -04:00
Alex Dadgar
bcb1a2e216 Do not allow path to escape the alloc dir for the FS commands 2016-10-03 14:58:44 -07:00
Diptanu Choudhury
c29861b418 Getting snapshot of allocation from remote node (#1741)
* Added the alloc dir move

* Moving allocdirs when starting allocations

* Added the migrate flag to ephemeral disk

* Stopping migration if the allocation doesn't need migration any more

* Added the GetAllocDir method

* refactored code

* Added a test for alloc runner

* Incorporated review comments
2016-10-03 09:59:57 -07:00
Diptanu Choudhury
bc82dd67b5 Closing files when files are removed 2016-09-23 22:17:53 -07:00
Diptanu Choudhury
dd3af85114 Adding a snapshot endpoint on the client (#1730) 2016-09-21 21:28:12 -07:00
Alex Dadgar
bec6adb2ee Vault token threaded 2016-09-14 13:30:01 -07:00
Alex Dadgar
4bae54741e Secret dir materialized in alloc/task directory 2016-09-02 12:44:05 -07:00
Alex Dadgar
335cababe1 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar
0b07ef93c1 Address comments and reserve 2016-08-31 18:11:02 -07:00
Alex Dadgar
46ce8dd020 Interface + tests 2016-08-30 21:40:32 -07:00
Alex Dadgar
0ebdff2bb7 Secret dir, hello world 2016-08-29 15:41:52 -07:00
Alex Dadgar
b813362ece Make maxSize exported so that it is serialized 2016-08-28 17:48:35 -07:00
Ivo Verberk
0d13488f1f Monitor the complete alloc directory, not just the shared part. 2016-08-25 20:48:19 +02:00
Ivo Verberk
13071e9846 Disk resource monitoring and enforcement 2016-08-18 07:59:03 +02:00
Alex Dadgar
17b9e7ddb4 only use polling 2016-08-11 18:59:48 -07:00
Alex Dadgar
cc0fec53f1 initial log api impl 2016-07-25 11:16:01 -07:00
Alex Dadgar
19ca6e7b55 address comments 2016-07-12 17:01:33 -06:00
Alex Dadgar
3cc13aeca1 initial comments 2016-07-11 10:58:18 -06:00
Alex Dadgar
2368722146 initial tail impl 2016-07-10 13:57:04 -04:00
Sean Chittenden
29be52f762 Speling police 2016-05-15 09:41:34 -07:00
Diptanu Choudhury
dc38906809 Removing directories only if we could successfully unmount them 2016-05-10 14:57:16 -07:00
Diptanu Choudhury
2e3a2d6e0d Seeking to offset while doing readAt with non-zero offset 2016-04-04 13:05:02 -07:00
Alex Dadgar
8742390da5 Alloc dir uses MkdirAll 2016-03-28 14:33:53 -07:00
Alex Dadgar
9c45f36819 Drop the permissions of sub directories in the alloc dir 2016-02-25 11:49:43 -08:00
Diptanu Choudhury
54d7057912 Writing logs to alloc dir 2016-02-24 20:06:43 -08:00
Alex Dadgar
1a31340075 Unmount special directories when task finishes 2016-02-08 18:51:11 -08:00
Alex Dadgar
ec583e4807 Add tmp file to each task directory 2016-02-04 15:35:04 -08:00