kemko/nomad
1
0
Fork 0
mirror of https://github.com/kemko/nomad.git synced 2026-01-01 16:05:42 +03:00
Code Issues Packages Projects Releases Wiki Activity

Workload Identity, Task API, and Dynamic Node Metadata Docs (#16102)

Browse Source 
* docs: add dynamic node metadata api docs

Also update all paths in the client API docs to explicitly state the
`/v1/` prefix. We're inconsistent about that, but I think it's better to
display the full path than to only show the fragment. If we ever do a
`/v2/` whether or not we explicitly state `/v1/` in our docs won't be
our greatest concern.

* docs: add task-api docs
This commit is contained in:
Michael Schurter
2023-02-09 16:03:43 -08:00
committed by GitHub
parent 05f6fbc7a1
commit eabb47e2d0
6 changed files with 345 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

272
website/content/api-docs/client.mdx
View File

@@ -2,13 +2,13 @@
layout: api
page_title: Client - HTTP API
description: |-
The /client endpoints are used to access client statistics and inspect
The /client endpoints are used to access client information and inspect
allocations running on a particular client.
---
# Client HTTP API
The `/client` endpoints are used to interact with the Nomad clients.
The `/v1/client` endpoints are used to interact with the Nomad clients.
Since Nomad 0.8.0, both a client and server can handle client endpoints. This is
particularly useful for when a direct connection to a client is not possible due
@@ -20,15 +20,179 @@ When accessing the endpoints via the server, if the desired node is ambiguous
based on the URL, an additional `?node_id` query parameter must be provided to
disambiguate.
## Read Node Metadata
This endpoint queries Node metadata on a specific Client agent and responds
with the following fields:
- `Meta` `(object)` - The Node metadata that will be registered with the Nomad
servers and used by the scheduler (after up to 10 seconds of delay for
batching). This is the merged version of the `Static` and `Dynamic` fields.
- `Static` `(object)` - The Node metadata set in the Client agent's
configuration file. Only loaded when an agent starts.
- `Dynamic` `(object)` - The Node metadata set via the API (see below). Unlike
`Meta` and `Static`, this object may contain `null` values to differentiate
"unset" keys from keys with an empty string value (`""`).
Note that [`/v1/node/:node_id`][api-node-read] only contains the `Meta` object.
It may take up to 10 seconds for dynamic Node metadata to be sent to Servers
and visible through the Node API. Use the Node API to see the version of Node
metadata the scheduler uses.
| Method | Path | Produces |
| ------ | --------------------- | ------------------ |
| `GET` | `/v1/client/metadata` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
[required ACLs](/nomad/api-docs#acls).
| Blocking Queries | ACL Required |
| ---------------- | ------------- |
| `NO` | `node:read` |
### Parameters
- `:node_id` `(string: <optional>)` - Specifies the node to query.
This is required when the endpoint is being accessed via a server. Defaults
to the node recieving the request otherwise. This is specified as part of
the URL. Note, this must be the _full_ node ID, not the short 8-character
one. This must be specified as part of the path (`?node_id=...`).
### Sample Request
```shell-session
$ nomad operator api /v1/client/metadata
```
### Sample Response
Formatted by appending `?pretty` above.
```json
{
"Meta": {
"connect.proxy_concurrency": "1",
"connect.sidecar_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.gateway_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.log_level": "debug",
"foo": "bar"
},
"Dynamic": {
"key_to_unset": null,
"foo": "bar",
"connect.log_level": "debug"
},
"Static": {
"connect.sidecar_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.gateway_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.log_level": "info",
"connect.proxy_concurrency": "1"
}
}
```
### Sample Request
## Update Node Metadata
This endpoint updates dynamic Node metadata on a specific Client agent. Since
dynamic Node metadata is only periodically synchronized to Nomad Servers, the
`Meta` returned in this API may not be reflected in the
[`/v1/node/:node_id`][api-node-read] API for up to 10 seconds. Scheduling uses
the Node API version of `Meta`.
For convenience this endpoint returns the same response as a GET.
| Method | Path | Produces |
| ------ | --------------------- | ------------------ |
| `POST` | `/v1/client/metadata` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
[required ACLs](/nomad/api-docs#acls).
| Blocking Queries | ACL Required |
| ---------------- | ------------- |
| `NO` | `node:write` |
### Parameters
- `NodeID` or `:node_id` `(string: <optional>)` - Specifies the node to query.
This is required when the endpoint is being accessed via a server. Defaults
to the node recieving the request otherwise. This is specified as part of
the URL. Note, this must be the _full_ node ID, not the short 8-character
one. This may be specified as part of the path (`?node_id=...`) or request
(`NodeID: "..."`).
- `Meta` `(object: <required>)` - Specifies the Node metadata keys to update.
Only specified keys are updated.
- `<key>` `(string: <optional>)` - Specifies a metadata key to update to a
particular value. Since `""` is a valid value and distinct from unset,
the `null` value is used to mark a key as unset. Keys must be valid
dotted HCL identifiers. For example `connect.log_level` is a valid key
while `some/path` is not.
### Sample Payload
```json
{
"Meta": {
"connect.log_level": "debug",
"key_to_unset": null,
"foo": "bar"
}
}
```
### Sample Request
Assuming the above payload is in a file called `meta.json`.
```shell-session
$ nomad operator api /v1/client/metadata < meta.json
```
### Sample Response
Formatted by appending `?pretty` above.
```json
{
"Meta": {
"connect.proxy_concurrency": "1",
"connect.sidecar_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.gateway_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.log_level": "debug",
"foo": "bar"
},
"Dynamic": {
"key_to_unset": null,
"foo": "bar",
"connect.log_level": "debug"
},
"Static": {
"connect.sidecar_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.gateway_image": "envoyproxy/envoy:v${NOMAD_envoy_version}",
"connect.log_level": "info",
"connect.proxy_concurrency": "1"
}
}
```
## Read Stats
This endpoint queries the actual resources consumed on a node. The API endpoint
is hosted by the Nomad client and requests have to be made to the nomad client
whose resource usage metrics are of interest.
| Method | Path | Produces |
| ------ | --------------- | ------------------ |
| `GET` | `/client/stats` | `application/json` |
| Method | Path | Produces |
| ------ | ------------------ | ------------------ |
| `GET` | `/v1/client/stats` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -48,8 +212,7 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/stats
$ nomad operator api /v1/client/stats
```
### Sample Response
@@ -203,9 +366,9 @@ $ curl \
The client `allocation` endpoint is used to query the actual resources consumed
by an allocation.
| Method | Path | Produces |
| ------ | ------------------------------------ | ------------------ |
| `GET` | `/client/allocation/:alloc_id/stats` | `application/json` |
| Method | Path | Produces |
| ------ | --------------------------------------- | ------------------ |
| `GET` | `/v1/client/allocation/:alloc_id/stats` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -224,8 +387,8 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/allocation/5fc98185-17ff-26bc-a802-0c74fa471c99/stats
$ nomad operator api \
/v1/client/allocation/5fc98185-17ff-26bc-a802-0c74fa471c99/stats
```
### Sample Response
@@ -286,9 +449,9 @@ $ curl \
This endpoint reads the contents of a file in an allocation directory.
| Method | Path | Produces |
| ------ | -------------------------- | ------------ |
| `GET` | `/client/fs/cat/:alloc_id` | `text/plain` |
| Method | Path | Produces |
| ------ | ----------------------------- | ------------ |
| `GET` | `/v1/client/fs/cat/:alloc_id` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -310,13 +473,13 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/cat/5fc98185-17ff-26bc-a802-0c74fa471c99
$ nomad operator api \
/v1/client/fs/cat/5fc98185-17ff-26bc-a802-0c74fa471c99
```
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/cat/5fc98185-17ff-26bc-a802-0c74fa471c99?path=alloc/file.json
$ nomad operator api \
/v1/client/fs/cat/5fc98185-17ff-26bc-a802-0c74fa471c99?path=alloc/file.json
```
### Sample Response
@@ -330,9 +493,9 @@ $ curl \
This endpoint reads the contents of a file in an allocation directory at a
particular offset and limit.
| Method | Path | Produces |
| ------ | ----------------------------- | ------------ |
| `GET` | `/client/fs/readat/:alloc_id` | `text/plain` |
| Method | Path | Produces |
| ------ | -------------------------------- | ------------ |
| `GET` | `/v1/client/fs/readat/:alloc_id` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -360,8 +523,8 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/readat/5fc98185-17ff-26bc-a802-0c74fa471c99?path=/alloc/foo&offset=1323&limit=19303
$ nomad operator api \
/v1/client/fs/readat/5fc98185-17ff-26bc-a802-0c74fa471c99?path=/alloc/foo&offset=1323&limit=19303
```
### Sample Response
@@ -374,9 +537,9 @@ $ curl \
This endpoint streams the contents of a file in an allocation directory.
| Method | Path | Produces |
| ------ | ----------------------------- | ------------ |
| `GET` | `/client/fs/stream/:alloc_id` | `text/plain` |
| Method | Path | Produces |
| ------ | -------------------------------- | ------------ |
| `GET` | `/v1/client/fs/stream/:alloc_id` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -406,8 +569,8 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/stream/5fc98185-17ff-26bc-a802-0c74fa471c99?path=/alloc/logs/redis.log
$ nomad operator api \
/v1/client/fs/stream/5fc98185-17ff-26bc-a802-0c74fa471c99?path=/alloc/logs/redis.log
```
### Sample Response
@@ -442,9 +605,9 @@ fields:
This endpoint streams a task's stderr/stdout logs.
| Method | Path | Produces |
| ------ | --------------------------- | ------------ |
| `GET` | `/client/fs/logs/:alloc_id` | `text/plain` |
| Method | Path | Produces |
| ------ | ------------------------------ | ------------ |
| `GET` | `/v1/client/fs/logs/:alloc_id` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -479,8 +642,8 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/logs/5fc98185-17ff-26bc-a802-0c74fa471c99
$ nomad operator api \
/v1/client/fs/logs/5fc98185-17ff-26bc-a802-0c74fa471c99
```
### Sample Response
@@ -515,9 +678,9 @@ fields:
This endpoint lists files in an allocation directory.
| Method | Path | Produces |
| ------ | ------------------------- | ------------ |
| `GET` | `/client/fs/ls/:alloc_id` | `text/plain` |
| Method | Path | Produces |
| ------ | ---------------------------- | ------------ |
| `GET` | `/v1/client/fs/ls/:alloc_id` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -539,8 +702,8 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/ls/5fc98185-17ff-26bc-a802-0c74fa471c99
$ nomad operator api \
/v1/client/fs/ls/5fc98185-17ff-26bc-a802-0c74fa471c99
```
### Sample Response
@@ -568,9 +731,9 @@ $ curl \
This endpoint stats a file in an allocation.
| Method | Path | Produces |
| ------ | --------------------------- | ------------ |
| `GET` | `/client/fs/stat/:alloc_id` | `text/plain` |
| Method | Path | Produces |
| ------ | ------------------------------ | ------------ |
| `GET` | `/v1/client/fs/stat/:alloc_id` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -592,8 +755,8 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/fs/stat/5fc98185-17ff-26bc-a802-0c74fa471c99
$ nomad operator api \
/v1/client/fs/stat/5fc98185-17ff-26bc-a802-0c74fa471c99
```
### Sample Response
@@ -614,9 +777,9 @@ This endpoint forces a garbage collection of a particular, stopped allocation
on a node. Note that the allocation will still exist on the server and appear
in server responses.
| Method | Path | Produces |
| ------ | --------------------------------- | ------------------ |
| `GET` | `/client/allocation/:alloc_id/gc` | `application/json` |
| Method | Path | Produces |
| ------ | ------------------------------------ | ------------------ |
| `GET` | `/v1/client/allocation/:alloc_id/gc` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -635,17 +798,17 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://nomad.rocks/v1/client/allocation/5fc98185-17ff-26bc-a802-0c74fa471c99/gc
$ nomad operator api \
/v1/client/allocation/5fc98185-17ff-26bc-a802-0c74fa471c99/gc
```
## GC All Allocation
This endpoint forces a garbage collection of all stopped allocations on a node.
| Method | Path | Produces |
| ------ | ------------ | ------------ |
| `GET` | `/client/gc` | `text/plain` |
| Method | Path | Produces |
| ------ | --------------- | ------------ |
| `GET` | `/v1/client/gc` | `text/plain` |
The table below shows this endpoint's support for
[blocking queries](/nomad/api-docs#blocking-queries) and
@@ -665,6 +828,7 @@ The table below shows this endpoint's support for
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/client/gc
$ nomad operator api /v1/client/gc
```
[api-node-read]: /nomad/api-docs/nodes

100
website/content/api-docs/task-api.mdx Normal file
View File

@@ -0,0 +1,100 @@
---
layout: api
page_title: Task HTTP API
description: |-
Jobs can access Nomad's HTTP API via the Task API.
---
# Task API
Nomad's Task API provides every task managed by Nomad with a Unix Domain Socket
(UDS) to access the local agent's HTTP API. Regardless of agent configuration
the Task API does *not* require [mTLS][], but *always* requires authentication.
See below for details.
The Unix Domain Socket is located at `${SECRETS_DIR}/api.sock`.
## Rationale
Nomad's HTTP API is available on every agent at the configured
[`bind_addr`][bind_addr]. While this is convenient for user access, it is not
always accessible to workloads running on Nomad. These workloads may have a
network configuration that makes it impossible to access the agent HTTP
address, or the agent's HTTP address may be difficult for workloads to discover
in a way that's portable between Nomad nodes and clusters.
A Unix Domain Socket is a way to expose network services that works with most
runtimes and operating systems and adds minimal complexity or runtime overhead
to Nomad.
## Security
Unlike the agent's HTTP API, the Task API *always requires authentication* even
if [ACLs][acl] are disabled. This allows Nomad to always make the Task API
available even if the workload is untrusted.
Both [ACL Tokens][acl-tokens] and [Workload Identities][workload-id] are
accepted. Once the Task API has authneticated the credentials, the normal
endpoint-specific authorization is applied when ACLs are enabled.
The Workload Identity should be used by tasks accessing the Task API.
An ACL Token should be used when an operator is accessing the Task API via
[`nomad alloc exec`][alloc-exec] or when a task is proxying Nomad HTTP requests
on behalf of an authenticated user. The Task API could be used by a proxy
presenting Nomad's UI with a standard TLS certificate for browsers.
If [`task.user`][task-user] is set in the jobspec, the Task API will only be
usable by that user. Otherwise the Unix Domain Socket is accessible by any
user.
mTLS is never enabled for the Task API since traffic never leaves the node.
## Using the Task API
The following jobspec will use the Task API to set [Dynamic Node Metadata][dnm]
and exit.
```hcl
job "taskapi-example" {
type = "batch"
group "taskapi-example" {
task "taskapi" {
driver = "docker"
config {
image = "curlimages/curl:7.87.0"
args = [
"--unix-socket", "${NOMAD_SECRETS_DIR}/api.sock",
"-H", "Authorization: Bearer ${NOMAD_TOKEN}",
"--data-binary", "{\"Meta\": {\"example\": \"Hello World!\"}}",
"--fail-with-body",
"--verbose",
"localhost/v1/client/metadata",
]
}
identity {
env = true
}
}
}
}
```
If the job was able to run successfully after about 10 seconds you can observe
the outcome by searching for the updated Node's metadata:
```shell-session
$ nomad node status -filter 'Meta.example == "Hello World!"'
```
[acl]: /nomad/docs/concepts/acl
[acl-tokens]: /nomad/docs/concepts/acl#token
[alloc-exec]: /nomad/docs/commands/alloc/exec
[bind_addr]: /nomad/docs/configuration
[mTLS]: /nomad/tutorials/transport-security/security-enable-tls
[task-user]: /nomad/docs/job-specification/task#user
[workload-id]: /nomad/docs/concepts/workload-identity

18
website/content/docs/concepts/workload-identity.mdx
View File

@@ -26,6 +26,23 @@ includes the following identity claims:
While Nomad always creates and uses workload identities internally, the JWT is
not exposed to tasks by default.
To expose Workload Identity to tasks, add an [`identity`][identity-block] block
to your jobspec:
```hcl
task "example" {
identity {
# Expose Workload Identity in NOMAD_TOKEN env var
env = true
# Expose Workload Identity in ${SECRETS_DIR}/nomad_token file
file = true
}
}
```
# Workload Associated ACL Policies
You can associate additional ACL policies with workload identities by passing
@@ -79,6 +96,7 @@ In Nomad 1.4.0 the workload identity is used only for `template`
access to [Variables][] and not exposed outside of Nomad.
[allocation]: /nomad/docs/concepts/architecture#allocation
[identity-block]: /nomad/docs/job-specification/identity
[plan applier]: /nomad/docs/concepts/scheduling/scheduling
[Variables]: /nomad/docs/concepts/variables
[JSON Web Token (JWT)]: https://datatracker.ietf.org/doc/html/rfc7519

4
website/content/docs/job-specification/identity.mdx
View File

@@ -47,9 +47,5 @@ job "docs" {
readable by that user. Otherwise the file is readable by everyone but is
protected by parent directory permissions.
Note that while both parameters default to `true`, the `identity` block itself
must be present in the job specification or the workload identity will not be
exposed.
[taskuser]: /nomad/docs/job-specification/task#user "Nomad task Block"
[Workload Identity]: /nomad/docs/concepts/workload-identity "Nomad Workload Identity"

5
website/content/docs/job-specification/task.mdx
View File

@@ -51,6 +51,9 @@ job "docs" {
- `env` <code>([Env][]: nil)</code> - Specifies environment variables that will
be passed to the running process.
- `identity` <code>([Identity][]: nil)</code> - Expose [Workload Identity][] to
the task.
- `kill_timeout` `(string: "5s")` - Specifies the duration to wait for an
application to gracefully quit before force-killing. Nomad first sends a
[`kill_signal`][kill_signal]. If the task does not exit before the configured
@@ -207,6 +210,7 @@ task "server" {
[affinity]: /nomad/docs/job-specification/affinity 'Nomad affinity Job Specification'
[dispatchpayload]: /nomad/docs/job-specification/dispatch_payload 'Nomad dispatch_payload Job Specification'
[env]: /nomad/docs/job-specification/env 'Nomad env Job Specification'
[Identity]: /nomad/docs/job-specification/identity 'Nomad identity Job Specification'
[meta]: /nomad/docs/job-specification/meta 'Nomad meta Job Specification'
[resources]: /nomad/docs/job-specification/resources 'Nomad resources Job Specification'
[lifecycle]: /nomad/docs/job-specification/lifecycle 'Nomad lifecycle Job Specification'
@@ -224,3 +228,4 @@ task "server" {
[user_denylist]: /nomad/docs/configuration/client#user-denylist
[max_kill]: /nomad/docs/configuration/client#max_kill_timeout
[kill_signal]: /nomad/docs/job-specification/task#kill_signal
[Workload Identity]: /nomad/docs/concepts/workload-identity 'Nomad Workload Identity'

4
website/data/api-docs-nav-data.json
View File

@@ -14,6 +14,10 @@
"title": "JSON Jobs",
"path": "json-jobs"
},
{
"title": "Task API",
"path": "task-api"
},
{
"divider": true
},