Commit Graph

19679 Commits

Author SHA1 Message Date
Michael Schurter
f44c04ecd1 s/0.13/1.0/g
1.0 here we come!
2020-10-14 15:17:47 -07:00
Michael Schurter
116b2b8b35 Merge pull request #9055 from hashicorp/f-9017-resources
api: add field filters to /v1/{allocations,nodes}
2020-10-14 14:49:39 -07:00
Michael Schurter
0695801256 unify boolean parameter parsing 2020-10-14 12:23:25 -07:00
Dave May
71a022ad8c Metrics gotemplate support, debug bundle features (#9067)
* add goroutine text profiles to nomad operator debug

* add server-id=all to nomad operator debug

* fix bug from changing metrics from string to []byte

* Add function to return MetricsSummary struct, metrics gotemplate support

* fix bug resolving 'server-id=all' when no servers are available

* add url to operator_debug tests

* removed test section which is used for future operator_debug.go changes

* separate metrics from operator, use only structs from go-metrics

* ensure parent directories are created as needed

* add suggested comments for text debug pprof

* move check down to where it is used

* add WaitForFiles helper function to wait for multiple files to exist

* compact metrics check

Co-authored-by: Drew Bailey <2614075+drewbailey@users.noreply.github.com>

* fix github's silly apply suggestion

Co-authored-by: Drew Bailey <2614075+drewbailey@users.noreply.github.com>
2020-10-14 15:16:10 -04:00
Tim Gross
b2fb40ec71 ci: make split between e2e and integration tests explicit (#9091)
Currently we compile (but don't run) the e2e tests as part of `test-other`,
which is skipped for branches named `e2e-*`. Move this check into the
`test-e2e` job. Split out the vault compatibility integration check as its own
makefile target for clarity.
2020-10-14 14:53:21 -04:00
Drew Bailey
d68e3d4218 Merge pull request #9013 from hashicorp/event-stream
Event stream
2020-10-14 14:45:34 -04:00
Drew Bailey
8c5f3a042d event stream changelog 2020-10-14 14:27:54 -04:00
Jimmy Merritello
349a62dafe [Website] Bump HSM version (#9090)
* Bump HSM version on website

* Updated pkg
2020-10-14 13:15:48 -05:00
Drew Bailey
3c15f41411 filter on additional filter keys, remove switch statement duplication
properly wire up durable event count

move newline responsibility

moves newline creation from NDJson to the http handler, json stream only encodes and sends now

ignore snapshot restore if broker is disabled

enable dev mode to access event steam without acl

use mapping instead of switch

use pointers for config sizes, remove unused ttl, simplify closed conn logic
2020-10-14 14:14:33 -04:00
Michael Schurter
276be8604a docs: document #9055 2020-10-14 10:41:32 -07:00
Michael Schurter
a55f46e9ba api: add field filters to /v1/{allocations,nodes}
Fixes #9017

The ?resources=true query parameter includes resources in the object
stub listings. Specifically:

- For `/v1/nodes?resources=true` both the `NodeResources` and
  `ReservedResources` field are included.
- For `/v1/allocations?resources=true` the `AllocatedResources` field is
  included.

The ?task_states=false query parameter removes TaskStates from
/v1/allocations responses. (By default TaskStates are included.)
2020-10-14 10:35:22 -07:00
Drew Bailey
71028207db namespace filtering 2020-10-14 12:44:43 -04:00
Drew Bailey
8711376e6a handle txn returning error 2020-10-14 12:44:42 -04:00
Drew Bailey
39ef3263ca Add EvictCallbackFn to handle removing entries from go-memdb when they
are removed from the event buffer.

Wire up event buffer size config, use pointers for structs.Events
instead of copying.
2020-10-14 12:44:42 -04:00
Drew Bailey
1288b18b27 rehydrate event publisher on snapshot restore
address pr feedback
2020-10-14 12:44:41 -04:00
Drew Bailey
66bda4455a go-memdb v1.3.0 2020-10-14 12:44:41 -04:00
Drew Bailey
8a3130a356 event durability count and cfg 2020-10-14 12:44:40 -04:00
Drew Bailey
004f634868 use Events to wrap index and events, store in events table 2020-10-14 12:44:39 -04:00
Drew Bailey
47d1a33eb5 writetxn can return error, add alloc and job generic events. Add events
table for durability
2020-10-14 12:44:39 -04:00
Drew Bailey
e7e6df394f wire up enable_event_publisher 2020-10-14 12:44:38 -04:00
Drew Bailey
8a57ee85f0 api comments 2020-10-14 12:44:38 -04:00
Drew Bailey
4f97bf8ef7 Events/eval alloc events (#9012)
* generic eval update event

first pass at alloc client update events

* api/event client
2020-10-14 12:44:37 -04:00
Drew Bailey
8d16846695 Events/deployment events (#9004)
* Node Drain events and Node Events (#8980)

Deployment status updates

handle deployment status updates (paused, failed, resume)

deployment alloc health

generate events from apply plan result

txn err check, slim down deployment event

one ndjson line per index

* consolidate down to node event + type

* fix UpdateDeploymentAllocHealth test invocations

* fix test
2020-10-14 12:44:37 -04:00
Drew Bailey
b825ba3bf1 Event Stream API/RPC (#8947)
This Commit adds an /v1/events/stream endpoint to stream events from.

The stream framer has been updated to include a SendFull method which
does not fragment the data between multiple frames. This essentially
treats the stream framer as a envelope to adhere to the stream framer
interface in the UI.

If the `encode` query parameter is omitted events will be streamed as
newline delimted JSON.
2020-10-14 12:44:36 -04:00
Drew Bailey
8bd15b534f Events/event source node (#8918)
* Node Register/Deregister event sourcing

example upsert node with context

fill in writetxnwithctx

ctx passing to handle event type creation, wip test

node deregistration event

drop Node from registration event

* node batch deregistration
2020-10-14 12:44:35 -04:00
Drew Bailey
26263091fd Events/cfg enable publisher (#8916)
* only enable publisher based on config

* add default prune tick

* back out state abandon changes on fsm close
2020-10-14 12:44:35 -04:00
Drew Bailey
c9bf8ce520 abandon current state on server shutdown 2020-10-14 12:44:34 -04:00
Drew Bailey
e7e2c799c8 Event Buffer Implemenation
adds an event buffer to hold events from raft changes.

update events to use event buffer

fix append call

provide way to prune buffer items after TTL

event publisher tests

basic publish test

wire up max item ttl

rename package to stream, cleanup exploratory work

subscription filtering

subscription plumbing

allow subscribers to consume events, handle closing subscriptions

back out old exploratory ctx work

fix lint

remove unused ctx bits

add a few comments

fix test

stop publisher on abandon
2020-10-14 12:44:34 -04:00
Chris Baker
38171b0dd6 Merge pull request #9089 from hashicorp/b-explicit-rune
fix go 1.15 pickiness
2020-10-14 10:37:36 -05:00
Tim Gross
b63d3ffc54 e2e: eliminate race condition causing rescheduling test flake (#9085)
The autorevert test checks for reverted allocations to be placed and running
before checking the deployment status, but the deployment can be completed and
marked "successful" before we check it for "running" status. Instead, just
wait for it to be marked "successful" and assert we have the expected count of
deployment statuses.
2020-10-14 11:35:30 -04:00
Tim Gross
554bdcc0ca e2e: use AMI filter for Ubuntu packer image (#9086)
Instead of hard-coding the base AMI for our Packer image for Ubuntu, use the
latest from Canonical so that we always have their current kernel patches.
2020-10-14 11:22:33 -04:00
Chris Baker
3245adb807 fix go 1.15 pickiness 2020-10-14 15:19:54 +00:00
Nick Ethier
5ddb88227e e2e/networking: use correct dc (#9088) 2020-10-14 11:14:09 -04:00
Seth Hoenig
b2edd17373 Merge pull request #9087 from hashicorp/docs-more-cl
docs: more connect changelog updates
2020-10-14 10:13:43 -05:00
Seth Hoenig
867ea69c93 docs: more connect changelog updates 2020-10-14 10:06:53 -05:00
Turtle Kalus
33cc888850 Use latest AMI for Ubuntu Xenial based on search (#9076)
Also, remove public share for ami build.
2020-10-14 11:01:54 -04:00
Tim Gross
b8cd187a41 e2e: add flag to opt-in to creating EBS/EFS volumes (#9082)
For everyday developer use, we don't need volumes for testing CSI. Providing a
flag to opt-in speeds up deploying dev clusters and slightly reduces infra costs.

Skip CSI test if missing volume specs.
2020-10-14 10:29:33 -04:00
Tim Gross
4314e81e78 E2E: vault secrets (#9081)
* rename vault API compatibility test for clarity
* exercise vault secrets lease renewal
2020-10-14 08:43:28 -04:00
Chris Baker
512fa09652 Merge pull request #9080 from hashicorp/f-remove-deprecated-metrics
removed backwards-compatible/untagged metrics deprecated in 0.7
2020-10-13 16:29:43 -05:00
Chris Baker
40c119c3d4 updated changelog for 9080 2020-10-13 20:18:39 +00:00
Chris Baker
797543ad4b removed backwards-compatible/untagged metrics deprecated in 0.7 2020-10-13 20:18:39 +00:00
Buck Doyle
74c14dfe55 Change htmlSafe prototype calls to imports (#9075) 2020-10-13 11:50:28 -05:00
Seth Hoenig
2214381825 Merge pull request #8945 from hashicorp/f-auto-sidecar
consul/connect: dynamically select envoy sidecar at runtime
2020-10-13 09:41:16 -05:00
Seth Hoenig
bdeb73cd2c consul/connect: dynamically select envoy sidecar at runtime
As newer versions of Consul are released, the minimum version of Envoy
it supports as a sidecar proxy also gets bumped. Starting with the upcoming
Consul v1.9.X series, Envoy v1.11.X will no longer be supported. Current
versions of Nomad hardcode a version of Envoy v1.11.2 to be used as the
default implementation of Connect sidecar proxy.

This PR introduces a change such that each Nomad Client will query its
local Consul for a list of Envoy proxies that it supports (https://github.com/hashicorp/consul/pull/8545)
and then launch the Connect sidecar proxy task using the latest supported version
of Envoy. If the `SupportedProxies` API component is not available from
Consul, Nomad will fallback to the old version of Envoy supported by old
versions of Consul.

Setting the meta configuration option `meta.connect.sidecar_image` or
setting the `connect.sidecar_task` stanza will take precedence as is
the current behavior for sidecar proxies.

Setting the meta configuration option `meta.connect.gateway_image`
will take precedence as is the current behavior for connect gateways.

`meta.connect.sidecar_image` and `meta.connect.gateway_image` may make
use of the special `${NOMAD_envoy_version}` variable interpolation, which
resolves to the newest version of Envoy supported by the Consul agent.

Addresses #8585 #7665
2020-10-13 09:14:12 -05:00
Tim Gross
405e9d874c docs: inclusive language configuration changes (#9069) 2020-10-13 08:02:29 -04:00
Seth Hoenig
d3a51279af Merge pull request #9038 from hashicorp/f-ec2-table
env_aws: get ec2 cpu perf data from AWS API
2020-10-12 18:55:33 -05:00
Seth Hoenig
1f782b3ce6 docs: tweak changelog ordering 2020-10-12 18:35:26 -05:00
Tim Gross
c1c09e1b65 Allow job Version to start at non-zero value (#9071)
Stop coercing version of new job to 0 in the state_store, so that we can add
regions to a multi-region deployment. Send new version, rather than existing
version, to MRD to accomodate version-choosing logic changes in ENT.

Co-authored-by: Chris Baker <1675087+cgbaker@users.noreply.github.com>
2020-10-12 13:59:48 -04:00
Nick Ethier
756aa11654 client: add NetworkStatus to Allocation (#8657) 2020-10-12 13:43:04 -04:00
Yoan Blanc
c14c616194 use allow/deny instead of the colored alternatives (#9019)
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-10-12 08:47:05 -04:00