kemko/nomad
1
0
Fork 0
mirror of https://github.com/kemko/nomad.git synced 2026-01-10 04:15:41 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,229 Commits 1 Branch 0 Tags
72df87b101fee71871bd00cd3de618dc592bee79
Commit Graph

13229 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mahmood Ali
72df87b101 address review comments 2018-11-20 13:21:39 -05:00
Mahmood Ali
10b073a81b add a missing no errorassertion 2018-11-19 21:44:00 -05:00
Mahmood Ali
cf51bc62e4 Fix a panic related to batch GC 
`deleteJobVersions` does concurrent modifications to iterated items
while iterating, by deleting job versions while it's iterating on them,
 2018-11-19 20:59:45 -05:00
Mahmood Ali
616ef214d9 Reproduce a panic related to batch GC 
Test case that reproduces a panic with the following stacktrace:

```
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1149715]

goroutine 35 [running]:
testing.tRunner.func1(0xc0001e2200)
	/usr/local/Cellar/go/1.11.2/libexec/src/testing/testing.go:792 +0x387
panic(0x167e400, 0x1c43a30)
	/usr/local/Cellar/go/1.11.2/libexec/src/runtime/panic.go:513 +0x1b9
github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-immutable-radix.(*Iterator).Next(0xc0003a4080, 0x17f7ba0, 0x0, 0xc0002e74a0, 0xc0003a0510, 0xc0003a0530, 0xc0003a0530)
	/go/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-immutable-radix/iter.go:81 +0xa5
github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.(*radixIterator).Next(0xc0003a0420, 0x1756059, 0xb)
	/go/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/txn.go:634 +0x2e
github.com/hashicorp/nomad/nomad/state.(*StateStore).deleteJobVersions(0xc00028f7d0, 0x2711, 0xc0002e7680, 0xc000392100, 0xc0003a4040, 0x0)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store.go:1130 +0x1a1
github.com/hashicorp/nomad/nomad/state.(*StateStore).DeleteJobTxn(0xc00028f7d0, 0x2711, 0x175334f, 0x7, 0xc000306810, 0x2f, 0xc000392100, 0x0, 0x0)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store.go:1102 +0x46c
github.com/hashicorp/nomad/nomad/state.TestStateStore_DeleteJobTxn_BatchDeletes.func1(0xc000392100, 0x1777ce0, 0xc000392100)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store_test.go:1705 +0x1a2
github.com/hashicorp/nomad/nomad/state.(*StateStore).WithWriteTransaction(0xc00028f7d0, 0xc0000d5e48, 0x0, 0x0)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store.go:3953 +0x79
github.com/hashicorp/nomad/nomad/state.TestStateStore_DeleteJobTxn_BatchDeletes(0xc0001e2200)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store_test.go:1703 +0x685
testing.tRunner(0xc0001e2200, 0x1777138)
	/usr/local/Cellar/go/1.11.2/libexec/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
	/usr/local/Cellar/go/1.11.2/libexec/src/testing/testing.go:878 +0x353
```
 2018-11-19 20:58:32 -05:00
Omar Khawaja
c5a222bd19 change instrumented webserver job file (#4897) 2018-11-17 01:10:44 -05:00
Omar Khawaja
dce594bb6d clarification regarding Nomad client generating token vs retrieving (#4890) 2018-11-16 19:52:31 -05:00
Mahmood Ali
6b7953af21 Merge pull request #4884 from hashicorp/f-alloc-devices-cli 
Report alloc device statistics in API and CLI
 2018-11-16 18:04:54 -05:00
Mahmood Ali
551a13b7b6 show Device Stats header in alloc status 2018-11-16 17:34:37 -05:00
Mahmood Ali
d8d0a55ee4 Show stable order of device attributes 2018-11-16 17:34:37 -05:00
Mahmood Ali
fd49039f09 address review comments 2018-11-16 17:13:01 -05:00
Michael Schurter
b0dcbb7665 Merge pull request #4843 from hashicorp/b-driver-env 
client: interpolate driver configurations
 2018-11-16 12:45:43 -06:00
Preetha
07e31669ba Update CHANGELOG.md 2018-11-16 12:31:54 -06:00
Preetha
5003f51dc9 Merge pull request #4889 from hashicorp/f-service-meta 
Pass service metadata "external-source" for consul UI integration
 2018-11-16 12:24:21 -06:00
Preetha Appan
3cf22d2903 Pass service metadata "external-source" for consul UI integration 2018-11-16 11:28:56 -06:00
Mahmood Ali
17eb416df7 tweak whitespace in device stats output 2018-11-16 10:37:39 -05:00
Mahmood Ali
4f18086d35 Display device stats in nomad alloc status 2018-11-16 10:26:32 -05:00
Mahmood Ali
6208d53275 Prepare to reuse device resources printing 2018-11-16 10:26:32 -05:00
Mahmood Ali
58cbafe913 Populate alloc stats API with device stats 
This change makes few compromises:

* Looks up the devices associated with tasks at look up time.  Given
that `nomad alloc status` is called rarely generally (compared to stats
telemetry and general job reporting), it seems fine.  However, the
lookup overhead grows bounded by number of `tasks x total-host-devices`,
which can be significant.

* `client.Client` performs the task devices->statistics lookup.  It
passes self to alloc/task runners so they can look up the device statistics
allocated to them.
  * Currently alloc/task runners are responsible for constructing the
entire RPC response for stats
  * The alternatives for making task runners device statistics aware
don't seem appealing (e.g. having task runners contain reference to hostStats)

* On the alloc aggregation resource usage, I did a naive merging of task device statistics.
  * Personally, I question the value of such aggregation, compared to
costs of struct duplication and bloating the response - but opted to be
consistent in the API.
  * With naive concatination, device instances from a single device group used by separate tasks in the alloc, would be aggregated in two separate device group statistics.
 2018-11-16 10:26:32 -05:00
Preetha
fff2d6f4d8 Merge pull request #4882 from sportebois/f-update-docs-env-stanza-coercion 
Change misleading boolean example in env stanza coercion section (#4820)
 2018-11-16 08:00:56 -06:00
Michael Schurter
9189173649 tests: fix tests post-rebase 2018-11-15 17:40:56 -08:00
Michael Schurter
a552a0c42c client/tr: add a bit of context to envbuilder errors 2018-11-15 16:26:25 -08:00
Michael Schurter
2283c8fff3 client: remove old proxy references from comments 2018-11-15 16:26:25 -08:00
Michael Schurter
926e3dc706 client: test more env key variations 2018-11-15 16:26:25 -08:00
Michael Schurter
b878bbf3f7 client: add new nested variables to task's hcl ctx 
The error messages are really bad, but it's extremely difficult to
produce good error messages without the original HCL.
 2018-11-15 16:26:25 -08:00
Michael Schurter
c4c0869a2e docs: document new interpolation behavior and options 2018-11-15 16:26:25 -08:00
Michael Schurter
26211408a6 client: turn env into nested objects for task configs 2018-11-15 16:25:57 -08:00
Michael Schurter
43b359914b client: interpolate driver configurations 
Also add missing SetDriverNetwork calls.
 2018-11-15 16:25:57 -08:00
Michael Schurter
ca4e237d62 gofmt -s -w command/helper_stats_test.go 
Fixes the static checks build
 2018-11-15 14:14:05 -08:00
Danielle Tomlinson
60c6cb8586 Merge pull request #4875 from hashicorp/f-constraints 
scheduler: Make != constraints more flexible
 2018-11-15 11:04:21 -08:00
Mahmood Ali
6542d6de75 Merge pull request #4879 from hashicorp/f-node-devices-cli 
Show device stats in nomad CLI
 2018-11-15 14:02:22 -05:00
Danielle Tomlinson
5aac3047e0 Add changelog entry for != operator 2018-11-15 11:00:32 -08:00
Danielle Tomlinson
cf8bc3224d docs: Add is_set/is_not_set 2018-11-15 11:00:32 -08:00
Danielle Tomlinson
3d0a45f6e5 scheduler: Add is_set/is_not_set constraints 
This adds constraints for asserting that a given attribute or value
exists, or does not exist. This acts as a companion to =, or !=
operators, e.g:

```hcl
constraint {
        attribute = "${attrs.type}"
        operator  = "!="
        value     = "database"
}

constraint {
        attribute = "${attrs.type}"
        operator  = "is_set"
}
```
 2018-11-15 11:00:32 -08:00
Sébastien Portebois
a71dea2ac9 Change misleading boolean example in env stanza coercion section 2018-11-15 13:12:58 -05:00
Mahmood Ali
5e2cdb48d4 Display StatsObject nested objects as well 2018-11-15 08:09:54 -05:00
Mahmood Ali
8274555d1c Use disk display format for devices 2018-11-14 22:13:23 -05:00
Mahmood Ali
85f5007043 Print verbose device in nomad node status -stats 2018-11-14 22:13:23 -05:00
Mahmood Ali
276149eaa7 device stats summary in node status 
Sample output with a mock device:

```
Host Resource Utilization
CPU             Memory          Disk
2651/26400 MHz  9.6 GiB/16 GiB  98 GiB/234 GiB

Device Resource Utilization
nomad/file/mock[README.md]    511 bytes
nomad/file/mock[e2e.go]       239 bytes
nomad/file/mock[e2e_test.go]  128 bytes

Allocations
No allocations placed
```
 2018-11-14 22:13:23 -05:00
Mahmood Ali
0ed1ab65b4 Merge pull request #4872 from hashicorp/f-devices-client-stats 
Expose Device Stats in Clients API
 2018-11-14 21:18:23 -05:00
Mahmood Ali
e102b3751f format vendor.json 2018-11-14 20:17:11 -05:00
Alex Dadgar
ab39343b91 Merge pull request #4874 from lunchbag/jen/update-share-img 
Update open graph image
 2018-11-14 14:36:44 -08:00
Mahmood Ali
c212716dda Add NodeResource Device types in api package 2018-11-14 14:42:36 -05:00
Mahmood Ali
04ecb5c72a Track Node Device attributes and serve them in API 2018-11-14 14:42:29 -05:00
Mahmood Ali
ba3fe15f7e Add Client Device Stats structs in api package 2018-11-14 14:41:19 -05:00
Mahmood Ali
5af9296bb4 Expose Device Stats in /client/stats API endpoint 2018-11-14 14:41:19 -05:00
Mahmood Ali
dd47c590f0 Allow nullable fields in StatValues 
In state values, we need to be able to distinguish between zero values
(e.g. `false`) and unset values (e.g. `nil`).

We can alternatively use protobuf `oneOf` and nested map to ensure
consistency of fields that are set together, but the golang
representation does not represent that well and introducing a mismatch
between representations.  Thus, I opted not to use it.
 2018-11-14 14:41:19 -05:00
Mahmood Ali
2f4c510cb7 Move Stat{Object|Value} to plugins/shared/structs 
Moving them as they may be useful for other packages/plugins besides
devices.
 2018-11-14 09:01:26 -05:00
Mahmood Ali
df694eb3be Regenerate proto files with protoc-gen-go@v1.2.0 2018-11-14 09:01:26 -05:00
Mahmood Ali
5697cbb183 fix comment typos 2018-11-14 08:36:14 -05:00
Omar Khawaja
3ad663be60 AWS sandbox environment upgrade (#4873) 
* upgrade Nomad from 0.8.4 to 0.8.6

* update deprecated nomad and vault commands

* update AMI ID

* add ingress rule for default fabio port and fabio UI

* upgrade Consul and Vault versions

* update AMI ID in README.md and terraform.tfvars
 2018-11-13 23:21:01 -05:00