* chore: prettify gutter-menu
* chore: add portal packages
* styling: add styles sidebar and portal behavior
* ui: sidebar component
* ui: create and implement statechart for evals
* ui: actor-relationship service and provider component
* ui: d3 hierarchy computation
* chore: add render-modifiers and curved arrows
* ui: create evaluation actor div
* fix related evaluations schema
* ui: register/deregister evaluation divs
* ui: handle resize behavior
* bug: infinite re-render cycle
* fix: conditional logic to prevent infinite render of flex resizing
* ui: related evaluations schema and request param
* ui: fix testing for evaluations
* refact: make related-evals a proper has-many
* chore: don't pauseTest
* temp: debug d3 hierarchy
* ui: move derived state logic into backing component class for detail
* ui: deprecated related evaluations logic in statechart
* ui: update evaluation models
* ui: update logic to paint svg in non-viewable scroll region
* ui: update styling
* ui: testing for eval detail view
* ui: delete detail from template directory
* ui: break detail component down
* ui: static data for /evaluation/:id endpoint
* ui: fix styling of d3 viz
* ui: add query parameter adapter for evals
* ui: last minute design requests
* wip: address browser updating detail view behavior
* refact: handle query-state change in statechart
* conditional class looking for currentEval equality (#12411)
* F UI/evaluation detail sidebar rel evals (#12415)
* ui: remove busy id alias from statechart
* ui: edit related evaluations viz error message
* ui: bug fixes on related evaluations view (#12423)
* ui: remove busy id alias from statechart
* ui: edit related evaluations viz error message
* ui: update error state
* ui: related evaluation outline styling
* Related evaluation stylefile and non-link if it matches the active sidebar (#12428)
* Adds tabbable and keyboard pressable evaluation table rows (#12433)
* ui: fix failing eval list tests (#12437)
* ui: move styling into classes (#12438)
* fix test failures (#12444)
* ui: move styling into classes
* ui: eslint disable
* ui: allocations have evaluations as async relationships
* ui: fix evaluation refresh button (#12447)
* ui: move styling into classes
* ui: eslint disable
* ui: allocations have evaluations as async relationships
* ui: refresh bug
* ui: final touches on sidebar (#12462)
* chore: turn off template linting rules
Temporarily turning off template linting because we dont have a set CSS convention and the release needs to go out ASAP.
* doc: deprecate out of date comments and vars
* ui: edit mirage server fetch logic
* ui: style sidebar relative
* Modification to mocked related evals and manually set 100% height on svg (#12460)
* F UI/evaluation detail sidebar final touches (#12463)
* chore: turn off template linting rules
Temporarily turning off template linting because we dont have a set CSS convention and the release needs to go out ASAP.
* doc: deprecate out of date comments and vars
* ui: edit mirage server fetch logic
* ui: style sidebar relative
* ui: account for new related eval added to chain
Co-authored-by: Michael Klein <michael@firstiwaslike.com>
Co-authored-by: Phil Renaud <phil@riotindustries.com>
Move some common Vault API data struct decoding out of the Vault client
so it can be reused in other situations.
Make Vault job validation its own function so it's easier to expand it.
Rename the `Job.VaultPolicies` method to just `Job.Vault` since it
returns the full Vault block, not just their policies.
Set `ChangeMode` on `Vault.Canonicalize`.
Add some missing tests.
Allows specifying an entity alias that will be used by Nomad when
deriving the task Vault token.
An entity alias assigns an indentity to a token, allowing better control
and management of Vault clients since all tokens with the same indentity
alias will now be considered the same client. This helps track Nomad
activity in Vault's audit logs and better control over Vault billing.
Add support for a new Nomad server configuration to define a default
entity alias to be used when deriving Vault tokens. This default value
will be used if the task doesn't have an entity alias defined.
When we unmount a volume we need to be able to recover from cases
where the plugin has been shutdown before the allocation that needs
it, so in #11892 we blocked shutting down the alloc runner hook. But
this blocks client shutdown if we're in the middle of unmounting. The
client won't be able to communicate with the plugin or send the
unpublish RPC anyways, so we should cancel the context and assume that
we'll resume the unmounting process when the client restarts.
For `-dev` mode we don't send the graceful `Shutdown()` method and
instead destroy all the allocations. In this case, we'll never be able
to communicate with the plugin but also never close the context we
need to prevent the hook from blocking. To fix this, move the retries
into their own goroutine that doesn't block the main `Postrun`.
The volume watcher design was based on deploymentwatcher and drainer,
but has an important difference: we don't want to maintain a goroutine
for the lifetime of the volume. So we stop the volumewatcher goroutine
for a volume when that volume has no more claims to free. But the
shutdown races with updates on the parent goroutine, and it's possible
to drop updates. Fortunately these updates are picked up on the next
core GC job, but we're most likely to hit this race when we're
replacing an allocation and that's the time we least want to wait.
Wait until the volume has "settled" before stopping this goroutine so
that the race between shutdown and the parent goroutine sending on
`<-updateCh` is pushed to after the window we most care about quick
freeing of claims.
* Fixes a resource leak when volumewatchers are no longer needed. The
volume is nil and can't ever be started again, so the volume's
`watcher` should be removed from the top-level `Watcher`.
* De-flakes the GC job test: the test throws an error because the
claimed node doesn't exist and is unreachable. This flaked instead of
failed because we didn't correctly wait for the first pass through the
volumewatcher.
Make the GC job wait for the volumewatcher to reach the quiescent
timeout window state before running the GC eval under test, so that
we're sure the GC job's work isn't being picked up by processing one
of the earlier claims. Update the claims used so that we're sure the
GC pass won't hit a node unpublish error.
* Adds trace logging to unpublish operations
Tear down the volume-consuming job between subtests, rather than after
all the tests are complete. For good measure, use a different ID for
the volume-consuming job as well.
In the same manner as the delete RPC, the list and read service
registration endpoints can be called either by external operators
or Nomad nodes. The latter occurs when a template is being
rendered which includes Nomad API template funcs. In this case,
the auth token is looked up as the node secret ID for auth.
* lint: require should not be aliased in core_sched_test
* lint: require should not be aliased in volumes_watcher_test
* testing: don't alias state package in core_sched_test
The client configuration options for drivers have been deprecated
since 0.9. We haven't torn them out completely but because they're
deprecated it's been hard to guarantee correct behavior. Remove the
documentation so that users aren't misled about their viability.
This is a test around upgrading from Nomad 0.8, which is long since
no longer supported. The test is slow, flaky, and imports consul/sdk.
Remove this test as it is no longer relevant.
These tend to fail on GHA, where I believe the client is not
starting up fast enough before making requests. So wait on
the client agent first.
```
=== RUN TestDebug_CapturedFiles
operator_debug_test.go:422: serverName: TestDebug_CapturedFiles.global, clientID, 1afb00e6-13f2-d8d6-d0f9-745a3fd6e8e4
operator_debug_test.go:492:
Error Trace: operator_debug_test.go:492
Error: Should be empty, but was No node(s) with prefix "1afb00e6-13f2-d8d6-d0f9-745a3fd6e8e4" found
Failed to retrieve clients, 0 nodes found in list: 1afb00e6-13f2-d8d6-d0f9-745a3fd6e8e4
Test: TestDebug_CapturedFiles
--- FAIL: TestDebug_CapturedFiles (0.08s)
```
* Wait longer for node to go down in disconnected clients test.
The existing helper only waits 10s, but there's a jitter on heartbeats
that we need to account for. Wait for 30s for node to go down to give
us plenty of room
* Port disconnected clients to stdlib-style test
In #12112 and #12113 we solved for the problem of races in releasing
volume claims, but there was a case that we missed. During a node
drain with a controller attach/detach, we can hit a race where we call
controller publish before the unpublish has completed. This is
discouraged in the spec but plugins are supposed to handle it
safely. But if the storage provider's API is slow enough and the
plugin doesn't handle the case safely, the volume can get "locked"
into a state where the provider's API won't detach it cleanly.
Check the claim before making any external controller publish RPC
calls so that Nomad is responsible for the canonical information about
whether a volume is currently claimed.
This has a couple side-effects that also had to get fixed here:
* Changing the order means that the volume will have a past claim
without a valid external node ID because it came from the client, and
this uncovered a separate bug where we didn't assert the external node
ID was valid before returning it. Fallthrough to getting the ID from
the plugins in the state store in this case. We avoided this
originally because of concerns around plugins getting lost during node
drain but now that we've fixed that we may want to revisit it in
future work.
* We should make sure we're handling `FailedPrecondition` cases from
the controller plugin the same way we handle other retryable cases.
* Several tests had to be updated because they were assuming we fail
in a particular order that we're no longer doing.
Resolves#12095 by WONTFIXing it.
This approach disables `writeToFile` as it allows arbitrary host
filesystem writes and is only a small quality of life improvement over
multiple `template` stanzas.
This approach has the significant downside of leaving people who have
altered their `template.function_denylist` *still vulnerable!* I added
an upgrade note, but we should have implemented the denylist as a
`map[string]bool` so that new funcs could be denied without overriding
custom configurations.
This PR also includes a bug fix that broke enabling all consul-template
funcs. We repeatedly failed to differentiate between a nil (unset)
denylist and an empty (allow all) one.
Concurrent E2E runs can collide when provisioning policies on HCP
Consul and HCP Vault. Namespace these by the test run name, as we do
for most everything else.
