Commit Graph

159 Commits

Author SHA1 Message Date
James Rasell
ca9e08e6b5 monitor: add log include location option on monitor CLI and API (#18795) 2023-10-20 07:55:22 +01:00
Charlie Voiselle
8a93ff3d2d [server] Directed leadership transfer CLI and API (#17383)
* Add directed leadership transfer func
* Add leadership transfer RPC endpoint
* Add ACL tests for leadership-transfer endpoint
* Add HTTP API route and implementation
* Add to Go API client
* Implement CLI command
* Add documentation
* Add changelog

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2023-10-04 12:20:27 -04:00
Phil Renaud
ef7bccbd40 [ui] ACL Roles in the UI, plus Role, Policy and Token management (#17770)
* Rename pages to include roles

* Models and adapters

* [ui] Any policy checks in the UI now check for roles' policies as well as token policies (#18346)

* combinedPolicies as a concept

* Classic decorator on role adapter

* We added a new request for roles, so the test based on a specific order of requests got fickle fast

* Mirage roles cluster scaffolded

* Acceptance test for roles and policies on the login page

* Update mirage mock for nodes fetch to account for role policies / empty token.policies

* Roles-derived policies checks

* [ui] Access Control with Roles and Tokens (#18413)

* top level policies routes moved into access control

* A few more routes and name cleanup

* Delog and test fixes to account for new url prefix and document titles

* Overview page

* Tokens and Roles routes

* Tokens helios table

* Add a role

* Hacky role page and deletion

* New policy keyboard shortcut and roles breadcrumb nav

* If you leave New Role but havent made any changes, remove the newly-created record from store

* Roles index list and general role route crud

* Roles index actually links to roles now

* Helios button styles for new roles and policies

* Handle when you try to create a new role without having any policies

* Token editing generally

* Create Token functionality

* Cant delete self-token but management token editing and deleting is fine

* Upgrading helios caused codemirror to explode, shimmed

* Policies table fix

* without bang-element condition, modifier would refire over and over

* Token TTL or Time setting

* time will take you on

* Mirage hooks for create and list roles

* Ensure policy names only use allow characters in mirage mocks

* Mirage mocked roles and policies in the default cluster

* log and lintfix

* chromedriver to 2.1.2

* unused unit tests removed

* Nice profile dropdown

* With the HDS accordion, rename our internal component scss ref

* design revisions after discussion

* Tooltip on deleted-policy tokens

* Two-step button peripheral isDeleting gcode removed

* Never to null on token save

* copywrite headers added and empty routefiles removed

* acceptance test fixes for policies endpoint

* Route for updating a token

* Policies testfixes

* Ember on-click-outside modifier upgraded with general ember-modifier upgrade

* Test adjustments to account for new profile header dropdown

* Test adjustments for tokens via policy pages

* Removed an unused route

* Access Control index page tests

* a11y tests

* Tokens index acceptance tests generally

* Lintfix

* Token edit page tests

* Token editing tests

* New token expiration tests

* Roles Index tests

* Role editing policies tests

* A complete set of Access Control Roles tests

* Policies test

* Be more specific about which row to check for expiration time

* Nil check on expirationTime equality

* Management tokens shouldnt show No Roles/Policies, give them their own designation

* Route guard on selftoken, conditional columns, and afterModel at parent to prevent orphaned policies on tokens/roles from stopping a new save

* Policy unloading on delete and other todos plus autofocus conditionally re-enabled

* Invalid policies non-links now a concept for Roles index

* HDS style links to make job.variables.alert links look like links again

* Mirage finding looks weird so making model async in hash even though redundant

* Drop rsvp

* RSVP wasnt the problem, cached lookups were

* remove old todo comments

* de-log
2023-09-27 14:53:09 -04:00
Daniel Bennett
fab968a748 csi: document volume expansion (#18573)
and show Capacity in `volume status` command.
2023-09-26 14:49:15 -05:00
Juana De La Cuesta
72acaf6623 [17449] Introduces a locking mechanism over variables (#18207)
It includes the work over the state store, the PRC server, the HTTP server, the go API package and the CLI's  command. To read more on the actuall functionality, refer to the RFCs [NMD-178] Locking with Nomad Variables and [NMD-179] Leader election using locking mechanism for the Autoscaler.
2023-09-21 17:56:33 +02:00
Gerard Nguyen
1339599185 cli: Add prune flag for nomad server force-leave command (#18463)
This feature will help operator to remove a failed/left node from Serf layer immediately
without waiting for 24 hours for the node to be reaped

* Update CLI with prune flag
* Update API /v1/agent/force-leave with prune query string parameter
* Update CLI and API doc
* Add unit test
2023-09-15 08:45:11 -04:00
Андрей Неустроев
3e61b3a37d Add multiple times in periodic jobs (#17858) 2023-08-22 15:42:31 -04:00
Luiz Aoqui
e21ab7d948 docs: fix job dispatch documentation (#18225) 2023-08-16 17:22:55 -04:00
Luiz Aoqui
01d71ca70e docs: expand documentation on node pools (#18109) 2023-08-16 11:16:06 -04:00
Shantanu Gadgil
a170499c32 docs: ampersand and bash backgrounding problem (#18175)
the `&` symbol messes up the command when copy pasting into a shell
2023-08-14 15:11:09 -04:00
Tim Gross
acfb4e679a docs: expand pprof documentation on goroutine profiles (#18172) 2023-08-08 08:33:42 -04:00
Tim Gross
902f640c80 docs: fix URL in agent pprof examples (#18142) 2023-08-03 16:05:53 -04:00
James Rasell
0a32d7ff5b docs: add allocation checks API documentation. (#18078) 2023-07-28 08:49:14 +01:00
Seth Hoenig
1e7726ce93 docs: note windows requirement for workload identity (#17950)
Support for UDS sockets was added to Windows 10.
2023-07-14 12:51:25 -05:00
Luiz Aoqui
f4c7182873 node pools: apply node pool scheduler configuration (#17598) 2023-06-21 20:31:50 -04:00
Luiz Aoqui
4f7c38b2a7 node pools: namespace integration (#17562)
Add structs and fields to support the Nomad Pools Governance Enterprise
feature of controlling node pool access via namespaces.

Nomad Enterprise allows users to specify a default node pool to be used
by jobs that don't specify one. In order to accomplish this, it's
necessary to distinguish between a job that explicitly uses the
`default` node pool and one that did not specify any.

If the `default` node pool is set during job canonicalization it's
impossible to do this, so this commit allows a job to have an empty node
pool value during registration but sets to `default` at the admission
controller mutator.

In order to guarantee state consistency the state store validates that
the job node pool is set and exists before inserting it.
2023-06-16 16:30:22 -04:00
Piotr Kazmierczak
be8f04e89f docs: corrections and additional information for OIDC-related concepts (#17470) 2023-06-09 16:50:22 +02:00
Piotr Kazmierczak
c1a9fe93ac docs: add missing login API endpoint documentation (#17467) 2023-06-09 15:59:01 +02:00
Tim Gross
9a6078a2ae node pools: implement support in scheduler (#17443)
Implement scheduler support for node pool:

* When a scheduler is invoked, we get a set of the ready nodes in the DCs that
  are allowed for that job. Extend the filter to include the node pool.
* Ensure that changes to a job's node pool are picked up as destructive
  allocation updates.
* Add `NodesInPool` as a metric to all reporting done by the scheduler.
* Add the node-in-pool the filter to the `Node.Register` RPC so that we don't
  generate spurious evals for nodes in the wrong pool.
2023-06-07 10:39:03 -04:00
Luiz Aoqui
354d741c95 node pool: implement nomad node pool nodes CLI (#17444) 2023-06-07 10:37:27 -04:00
Tim Gross
385dbfb8d1 node pools: implement HTTP API to list jobs in pool (#17431)
Implements the HTTP API associated with the `NodePool.ListJobs` RPC, including
the `api` package for the public API and documentation.

Update the `NodePool.ListJobs` RPC to fix the missing handling of the special
"all" pool.
2023-06-06 11:40:13 -04:00
Luiz Aoqui
f0f4cbb848 node pools: list nodes in pool (#17413) 2023-06-06 10:43:43 -04:00
Luiz Aoqui
637ddf516e node pools: add event stream support (#17412) 2023-06-06 10:14:47 -04:00
Luiz Aoqui
81f0b359dd node pools: register a node in a node pool (#17405) 2023-06-02 17:50:50 -04:00
Luiz Aoqui
9ee68fc02c node pool: add search support (#17385) 2023-06-01 17:48:14 -04:00
Tim Gross
2d059bbf22 node pools: add node_pool field to job spec (#17379)
This changeset only adds the `node_pool` field to the jobspec, and ensures that
it gets picked up correctly as a change. Without the rest of the implementation
landed yet, the field will be ignored.
2023-06-01 16:08:55 -04:00
Luiz Aoqui
970e998b00 node pools: add CRUD API (#17384) 2023-06-01 15:55:49 -04:00
Tim Gross
2aa3c746c4 logs: fix missing allocation logs after update to Nomad 1.5.4 (#17087)
When the server restarts for the upgrade, it loads the `structs.Job` from the
Raft snapshot/logs. The jobspec has long since been parsed, so none of the
guards around the default value are in play. The empty field value for `Enabled`
is the zero value, which is false.

This doesn't impact any running allocation because we don't replace running
allocations when either the client or server restart. But as soon as any
allocation gets rescheduled (ex. you drain all your clients during upgrades),
it'll be using the `structs.Job` that the server has, which has `Enabled =
false`, and logs will not be collected.

This changeset fixes the bug by adding a new field `Disabled` which defaults to
false (so that the zero value works), and deprecates the old field.

Fixes #17076
2023-05-04 16:01:18 -04:00
Seth Hoenig
7744caed48 connect: use explicit docker.io prefix in default envoy image names (#17045)
This PR modifies references to the envoyproxy/envoy docker image to
explicitly include the docker.io prefix. This does not affect existing
users, but makes things easier for Podman users, who otherwise need to
specify the full name because Podman does not default to docker.io
2023-05-02 09:27:48 -05:00
Tim Gross
30bc456f03 logs: allow disabling log collection in jobspec (#16962)
Some Nomad users ship application logs out-of-band via syslog. For these users
having `logmon` (and `docker_logger`) running is unnecessary overhead. Allow
disabling the logmon and pointing the task's stdout/stderr to /dev/null.

This changeset is the first of several incremental improvements to log
collection short of full-on logging plugins. The next step will likely be to
extend the internal-only task driver configuration so that cluster
administrators can turn off log collection for the entire driver.

---

Fixes: #11175

Co-authored-by: Thomas Weber <towe75@googlemail.com>
2023-04-24 10:00:27 -04:00
Luiz Aoqui
e169847a64 docs: add missing field Capabilities to Namespace API (#16931) 2023-04-19 08:14:36 -07:00
Luiz Aoqui
ba364fc400 docs: add missing API field JobACL and fix workload identity headers (#16930) 2023-04-19 08:12:58 -07:00
Seth Hoenig
2c44cbb001 api: enable support for setting original job source (#16763)
* api: enable support for setting original source alongside job

This PR adds support for setting job source material along with
the registration of a job.

This includes a new HTTP endpoint and a new RPC endpoint for
making queries for the original source of a job. The
HTTP endpoint is /v1/job/<id>/submission?version=<version> and
the RPC method is Job.GetJobSubmission.

The job source (if submitted, and doing so is always optional), is
stored in the job_submission memdb table, separately from the
actual job. This way we do not incur overhead of reading the large
string field throughout normal job operations.

The server config now includes job_max_source_size for configuring
the maximum size the job source may be, before the server simply
drops the source material. This should help prevent Bad Things from
happening when huge jobs are submitted. If the value is set to 0,
all job source material will be dropped.

* api: avoid writing var content to disk for parsing

* api: move submission validation into RPC layer

* api: return an error if updating a job submission without namespace or job id

* api: be exact about the job index we associate a submission with (modify)

* api: reword api docs scheduling

* api: prune all but the last 6 job submissions

* api: protect against nil job submission in job validation

* api: set max job source size in test server

* api: fixups from pr
2023-04-11 08:45:08 -05:00
Piotr Kazmierczak
3da1756dd0 acl: JWT changelog entry and typo fix 2023-03-30 09:40:11 +02:00
Piotr Kazmierczak
ed068c0831 acl: JWT auth method 2023-03-30 09:39:56 +02:00
James Rasell
96740b5392 docs: remove Java and Scala SDKs from supported list. (#16555) 2023-03-20 15:35:02 +01:00
Michael Schurter
46ae1025bb docs: dispatch_payload and jobs api docs had some weirdness (#16514)
* docs: dispatch_payload docs had some weirdness

Docs said "Examples" when there was only 1 example. Not sure what the
floating "to" in the description was for.

* docs: missing a heading level on jobs api docs
2023-03-16 09:42:46 -07:00
Luiz Aoqui
f2bfbfaf03 acl: update job eval requirement to submit-job (#16463)
The job evaluate endpoint creates a new evaluation for the job which is
a write operation. This change modifies the necessary capability from
`read-job` to `submit-job` to better reflect this.
2023-03-13 17:13:54 -04:00
Luiz Aoqui
158d6a91c2 docs: fix alloc stop no_shutdown_delay (#16282) 2023-03-03 14:44:49 -05:00
Aofei Sheng
64d27c6a2f docs: fix typos in task-api.mdx and workload-identity.mdx (#16309) 2023-03-03 08:37:59 -05:00
Dao Thanh Tung
41e510abe4 Fix missing query parameter in job doc (#16233)
Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>
2023-02-22 10:28:32 -06:00
Seth Hoenig
ed4ad3e7b7 docs: slight tidy up of var create example payload (#16212) 2023-02-17 13:12:39 -06:00
James Rasell
004ddb2b63 acl: add validation to binding rule selector on upsert. (#16210)
* acl: add validation to binding rule selector on upsert.

* docs: add more information on binding rule selector escaping.
2023-02-17 15:38:55 +01:00
Michael Schurter
037823e864 Minor post-1.5-beta1 API, code, and docs cleanups (#16193)
* api: return error on parse failure

* docs: clarify anonymous policy with task api
2023-02-16 10:32:21 -08:00
Michael Schurter
eabb47e2d0 Workload Identity, Task API, and Dynamic Node Metadata Docs (#16102)
* docs: add dynamic node metadata api docs

Also update all paths in the client API docs to explicitly state the
`/v1/` prefix. We're inconsistent about that, but I think it's better to
display the full path than to only show the fragment. If we ever do a
`/v2/` whether or not we explicitly state `/v1/` in our docs won't be
our greatest concern.

* docs: add task-api docs
2023-02-09 16:03:43 -08:00
Charlie Voiselle
fe4ff5be2a Add option to expose workload token to task (#15755)
Add `identity` jobspec block to expose workload identity tokens to tasks.

---------

Co-authored-by: Anders <mail@anars.dk>
Co-authored-by: Tim Gross <tgross@hashicorp.com>
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
2023-02-02 10:59:14 -08:00
Daniel Bennett
9f583f57f5 Change job init default to example.nomad.hcl and recommend in docs (#15997)
recommend .nomad.hcl for job files instead of .nomad (without .hcl)
* nomad job init -> example.nomad.hcl
* update docs
2023-02-02 11:47:47 -06:00
James Rasell
e4e4dc18e6 acl: fix a bug in token creation when parsing expiration TTLs. (#15999)
The ACL token decoding was not correctly handling time duration
syntax such as "1h" which forced people to use the nanosecond
representation via the HTTP API.

The change adds an unmarshal function which allows this syntax to
be used, along with other styles correctly.
2023-02-01 17:43:41 +01:00
Mike Nomitch
c2491e9146 Increases max variable size to 64KiB from 16KiB (#15983) 2023-01-31 13:32:36 -05:00
Piotr Kazmierczak
949a6f60c7 renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00