mirror of
https://github.com/kemko/nomad.git
synced 2026-01-07 10:55:42 +03:00
* Move commands from docs to its own root-level directory * temporarily use modified dev-portal branch with nomad ia changes * explicitly clone nomad ia exp branch * retrigger build, fixed dev-portal broken build * architecture, concepts and get started individual pages * fix get started section destinations * reference section * update repo comment in website-build.sh to show branch * docs nav file update capitalization * update capitalization to force deploy * remove nomad-vs-kubernetes dir; move content to what is nomad pg * job section * Nomad operations category, deploy section * operations category, govern section * operations - manage * operations/scale; concepts scheduling fix * networking * monitor * secure section * remote auth-methods folder and move up pages to sso; linkcheck * Fix install2deploy redirects * fix architecture redirects * Job section: Add missing section index pages * Add section index pages so breadcrumbs build correctly * concepts/index fix front matter indentation * move task driver plugin config to new deploy section * Finish adding full URL to tutorials links in nav * change SSO to Authentication in nav and file system * Docs NomadIA: Move tutorials into NomadIA branch (#26132) * Move governance and policy from tutorials to docs * Move tutorials content to job-declare section * run jobs section * stateful workloads * advanced job scheduling * deploy section * manage section * monitor section * secure/acl and secure/authorization * fix example that contains an unseal key in real format * remove images from sso-vault * secure/traffic * secure/workload-identities * vault-acl change unseal key and root token in command output sample * remove lines from sample output * fix front matter * move nomad pack tutorials to tools * search/replace /nomad/tutorials links * update acl overview with content from deleted architecture/acl * fix spelling mistake * linkcheck - fix broken links * fix link to Nomad variables tutorial * fix link to Prometheus tutorial * move who uses Nomad to use cases page; move spec/config shortcuts add dividers * Move Consul out of Integrations; move namespaces to govern * move integrations/vault to secure/vault; delete integrations * move ref arch to docs; rename Deploy Nomad back to Install Nomad * address feedback * linkcheck fixes * Fixed raw_exec redirect * add info from /nomad/tutorials/manage-jobs/jobs * update page content with newer tutorial * link updates for architecture sub-folders * Add redirects for removed section index pages. Fix links. * fix broken links from linkcheck * Revert to use dev-portal main branch instead of nomadIA branch * build workaround: add intro-nav-data.json with single entry * fix content-check error * add intro directory to get around Vercel build error * workound for emtpry directory * remove mdx from /intro/ to fix content-check and git snafu * Add intro index.mdx so Vercel build should work --------- Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
347 lines
13 KiB
Plaintext
347 lines
13 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Nomad dynamic host volume specification
|
|
description: |-
|
|
Learn about the Nomad dynamic host volume specification. Define capacity,
|
|
capability, constraint, node, node pool, and parameters passed directly
|
|
to the plugin to configure the volume. Learn how volume creation and
|
|
registration are different. Additionally, learn how to place a volume on
|
|
specific nodes, update a volume, and expand a volume's capacity.
|
|
---
|
|
|
|
# Nomad dynamic host volume specification
|
|
|
|
This page provides reference information for the Nomad dynamic host volume
|
|
specification. Create and register dynamic host volumes using the [`volume
|
|
create`][] and [`volume register`][] commands and the [`PUT
|
|
/v1/volume/host/create`][api_volume_create] and [`PUT
|
|
/v1/volume/host/register`][api_volume_register] API endpoints. Define capacity,
|
|
capability, constraint, node, node pool, and parameters passed directly to the
|
|
plugin to configure the volume. Learn how volume creation and registration are
|
|
different. Additionally, learn how to place a volume on specific nodes, update a
|
|
volume, and expand a volume's capacity.
|
|
|
|
Some attributes are only be supported by specific operation, while others may
|
|
have a different meaning for each action, so read the documentation for each
|
|
attribute carefully. The section [Differences Between Create and
|
|
Register](#differences-between-create-and-register) provides a summary of the
|
|
differences.
|
|
|
|
The file may be provided as either HCL or JSON to the commands and as JSON to
|
|
the API.
|
|
|
|
## Parameters
|
|
|
|
- `capacity` `(string: <optional>)` - The size of a volume in bytes. Either the
|
|
physical size of a disk or a quota, depending on the plugin. This field must
|
|
be between the `capacity_min` and `capacity_max` values unless they are
|
|
omitted. Accepts human-friendly suffixes such as `"100GiB"`. Only supported
|
|
for volume registration.
|
|
|
|
- `capacity_min` `(string: <optional>)` - Option for requesting a minimum
|
|
capacity, in bytes. The capacity of a volume may be the physical size of a
|
|
disk, or a quota, depending on the plugin. The specific size of the resulting
|
|
volume is somewhere between `capacity_min` and `capacity_max`; the exact
|
|
behavior is up to the plugin. If you want to specify an exact size, set
|
|
`capacity_min` and `capacity_max` to the same value. Accepts human-friendly
|
|
suffixes such as `"100GiB"`. Plugins that cannot restrict the size of volumes,
|
|
such as the built-in [`mkdir`][mkdir_plugin] plugin, may ignore this field.
|
|
|
|
- `capacity_max` `(string: <optional>)` - Option for requesting a maximum
|
|
capacity, in bytes. The capacity of a volume may be the physical size of a
|
|
disk, or a quota, depending on the plugin. The specific size of the resulting
|
|
volume is somewhere between `capacity_min` and `capacity_max`; the exact
|
|
behavior is up to the plugin. If you want to specify an exact size, set
|
|
`capacity_min` and `capacity_max` to the same value. Accepts human-friendly
|
|
suffixes such as `"100GiB"`. Plugins that cannot restrict the size of volumes,
|
|
such as the built-in [`mkdir`][mkdir_plugin] plugin, may ignore this field.
|
|
|
|
- `capability` <code>([Capability][capability]: <required>)</code> -
|
|
Option for validating the capability of a volume.
|
|
|
|
- `constraint` <code>([Constraint][constraint]: <optional>)</code> - A
|
|
restriction on the eligible nodes where a volume can be created. Refer to the
|
|
[volume placement](#volume-placement) section for details. You can provide
|
|
multiple `constraint` blocks to add more constraints. Optional for volume
|
|
creation and ignored for volume registration.
|
|
|
|
- `id` `(string: <optional>)` - The ID of a previously created volume to update
|
|
via `volume create` or `volume register`. You should never set this field when
|
|
initially creating or registering a volume, and you should only use the values
|
|
returned from the Nomad API for the ID.
|
|
|
|
- `host_path` `(string)` - The path on disk where the volume exists. You should
|
|
set this only for volume registration. It is ignored for volume creation.
|
|
|
|
- `name` `(string: <required>)` - The name of the volume, which is used as the
|
|
[`volume.source`][volume_source] field in job specifications that claim this
|
|
volume. Host volume names must be unique per node. Names are visible to any
|
|
user with `node:read` ACL, even across namespaces, so they should not be
|
|
treated as sensitive values.
|
|
|
|
- `namespace` `(string: <optional>)` - The namespace of the volume. This field
|
|
overrides the namespace provided by the `-namespace` flag or `NOMAD_NAMESPACE`
|
|
environment variable. Defaults to `"default"` if unset.
|
|
|
|
- `node_id` `(string)` - A specific node where you would like the volume to be
|
|
created. Refer to the [volume placement](#volume-placement) section for
|
|
details. Optional for volume creation but required for volume registration.
|
|
|
|
- `node_pool` `(string: <optional>)` - A specific node pool where you would like
|
|
the volume to be created. Refer to the [volume placement](#volume-placement)
|
|
section for details. Optional for volume creation or volume registration. If
|
|
you also provide `node_id`, the node must be in the provided `node_pool`.
|
|
|
|
- `parameters` <code>(map<string|string>:nil)</code> - An optional key-value map
|
|
of strings passed directly to the plugin to configure the volume. The details
|
|
of these parameters are specific to the plugin.
|
|
|
|
- `plugin_id` `(string)` - The ID of the [dynamic host volume
|
|
plugin][dhv_plugin] that manages this volume. Required for volume
|
|
creation. Nomad has one built-in plugin called [`mkdir`][mkdir_plugin].
|
|
|
|
- `type` `(string: <required>)` - The type of volume. Must be `"host"` for
|
|
dynamic host volumes.
|
|
|
|
## mkdir plugin
|
|
|
|
Nomad has one built-in plugin called `mkdir`, which creates a directory on the
|
|
host in the Nomad agent's [host_volumes_dir][]. The directory name is the
|
|
volume's ID.
|
|
|
|
`mkdir` ignores `capacity_min` and `capacity_max` volume configuration,
|
|
since it has no way of enforcing them.
|
|
|
|
### mkdir parameters
|
|
|
|
- `mode` `(string: <optional>)` - [Numeric
|
|
notation](https://en.wikipedia.org/wiki/File-system_permissions#Numeric_notation)
|
|
to apply to the created directory. Defaults to "0700", so only the owner can
|
|
access it. Must be a string, or you may get unexpected results.
|
|
- `uid` `(int: <optional>)` - User ID to own the directory. Defaults to the
|
|
user running the Nomad client agent, which is commonly `0` (root).
|
|
- `gid` `(int: <optional>)` - Group ID to own the directory. Defaults to the
|
|
user running the Nomad client agent, which is commonly `0` (root).
|
|
|
|
The user and group IDs must be present on any node that may receive the volume.
|
|
|
|
### mkdir example
|
|
|
|
<CodeBlockConfig filename="mkdir.volume.hcl">
|
|
|
|
```hcl
|
|
type = "host"
|
|
name = "cool-host-vol"
|
|
plugin_id = "mkdir"
|
|
parameters = {
|
|
mode = "0755"
|
|
uid = 1000
|
|
gid = 1000
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
## Differences between create and register
|
|
|
|
Several fields are set automatically by Nomad or the plugin when `volume create`
|
|
or `volume register` commands, or the equivalent APIs, are successful and you
|
|
should not set their values if they are not supported by the operation.
|
|
|
|
In **volume creation** you must set the [`plugin_id`](#plugin_id) field. The
|
|
[`capacity`](#capacity) and [`host_path`](#host_path) fields are ignored.
|
|
|
|
In **volume registration** you must set the [`node_id`](#node_id) and
|
|
[`host_path`](#host_path) fields. The [`plugin_id`](#plugin_id) and
|
|
[`constraint`](#constraint) fields are ignored. The [`node_pool`](#node_pool),
|
|
[`capacity_max`](#capacity_max), and [`capacity_min`](#capacity_min) fields are
|
|
ignored but must be consistent if set; the `node_pool` must match the node set
|
|
by `node_id`, and the minimum capacity must be less than the `capacity_max` and
|
|
`capacity`.
|
|
|
|
## Volume placement
|
|
|
|
The `volume create` command creates the volume on a single node. If `node_id` is
|
|
set, the volume is created on that node. The `node_pool` must be unset or match
|
|
that node. Otherwise, the Nomad server iterates over the available nodes and
|
|
place the volume on the first available node based on the following factors:
|
|
|
|
* The node cannot already have a host volume with the same name.
|
|
* If `node_pool` is set, the selected node must be in that node pool.
|
|
* The node must meet any and all constraints defined by the `constraint` fields.
|
|
|
|
## Update a volume definition
|
|
|
|
The `volume create` and `volume register` commands allow updating a volume
|
|
definition. However, after volume registration, you are only allowed to update
|
|
the following fields:
|
|
|
|
- `plugin_id`
|
|
- `capacity_min` and `capacity_max`. You may increase
|
|
the volume size if the plugin supports it. Expansion may or may not be
|
|
possible while the volume is in use, again depending on the plugin. Reducing
|
|
volume capacity is not allowed. Only available for volume creation.
|
|
- `capacity`, but only updated for volume registration.
|
|
- `constraint` fields. Ignored after the volume is created.
|
|
|
|
Additionally, you may add or remove `capability` blocks, but only if the
|
|
capability is not currently in use by a mounted volume.
|
|
|
|
You cannot update the `name`, `type`, and `node_id` fields. You may only change
|
|
the `node_pool` field from empty to the node pool that matches the `node_id`
|
|
field.
|
|
|
|
## Volume expansion
|
|
|
|
Dynamic host volumes may be expanded if the plugin allows. Reducing the size of
|
|
a volume is not permitted.
|
|
|
|
To trigger a volume expansion, increase [`capacity_min`](#capacity_min) above
|
|
the current real capacity of the volume (as shown with the [`volume status`][]
|
|
command), and re-issue [`volume create`][].
|
|
|
|
Nomad reconciles the requested capacity by issuing a create request to the
|
|
plugin.
|
|
|
|
## Examples
|
|
|
|
### Volume creation
|
|
|
|
This is an example file used for the [`volume create`][] command.
|
|
|
|
```hcl
|
|
name = "database"
|
|
type = "host"
|
|
plugin_id = "lvm-thin-provisioner"
|
|
node_pool = "prod"
|
|
|
|
capacity_min = "80G"
|
|
capacity_max = "100G"
|
|
|
|
capability {
|
|
access_mode = "single-node-reader-only"
|
|
attachment_mode = "file-system"
|
|
}
|
|
|
|
capability {
|
|
access_mode = "single-node-writer"
|
|
attachment_mode = "file-system"
|
|
}
|
|
|
|
parameters {
|
|
skuname = "Premium_LRS"
|
|
}
|
|
```
|
|
|
|
### Volume registration
|
|
|
|
This is an example file used for the [`volume register`][] command.
|
|
|
|
```hcl
|
|
name = "database"
|
|
type = "host"
|
|
node_id = "a7b4c0ca-cc78-11ef-8b5a-cb6ea67b844c"
|
|
host_path = "/var/srv/example"
|
|
capacity = "80G"
|
|
|
|
capability {
|
|
access_mode = "single-node-reader-only"
|
|
attachment_mode = "file-system"
|
|
}
|
|
|
|
capability {
|
|
access_mode = "single-node-writer"
|
|
attachment_mode = "file-system"
|
|
}
|
|
|
|
parameters {
|
|
skuname = "Premium_LRS"
|
|
}
|
|
```
|
|
|
|
### Volume expansion
|
|
|
|
This example shows how to expand a volume after it has been created or
|
|
registered.
|
|
|
|
1. Create a volume configuration file called `volume.hcl` that defines name,
|
|
type, plugin ID, and capacity.
|
|
|
|
```hcl
|
|
name = "database"
|
|
type = "host"
|
|
plugin_id = "external-plugin"
|
|
|
|
capacity_min = "30GiB"
|
|
capacity_max = "50GiB"
|
|
```
|
|
|
|
1. Create the volume using the [`volume create`] command.
|
|
|
|
```shell-session
|
|
$ nomad volume create ./volume.hcl
|
|
==> Created host volume database with ID 0c903229-311d-ba8a-f77e-45c31b83fab3
|
|
✓ Host volume "0c903229" ready
|
|
|
|
2025-01-06T16:56:09-05:00
|
|
ID = 0c903229-311d-ba8a-f77e-45c31b83fab3
|
|
Name = database
|
|
Namespace = default
|
|
Plugin ID = external-plugin
|
|
Node ID = cfe033a7-50de-2c46-cd18-12be7429eeb3
|
|
Node Pool = default
|
|
Capacity = 50 GiB
|
|
State = ready
|
|
Host Path = /run/nomad/dev/alloc_mounts/0c903229-311d-ba8a-f77e-45c31b83fab3
|
|
```
|
|
|
|
1. Review the volume's current capacity using the [`volume status`] command.
|
|
|
|
```shell-session
|
|
$ nomad volume status -type=host 0c903229 | grep Capacity
|
|
Capacity = 50 GiB
|
|
```
|
|
|
|
1. Increase volume capacity in the `volume.hcl` file.
|
|
|
|
Update the `capacity_min` and `capacity_max` fields.
|
|
|
|
```hcl
|
|
id = "0c903229-311d-ba8a-f77e-45c31b83fab3"
|
|
name = "database"
|
|
type = "host"
|
|
plugin_id = "external-plugin"
|
|
|
|
capacity_min = "100GiB" # double
|
|
capacity_max = "100GiB" # increased to match
|
|
```
|
|
|
|
1. Expand the volume using the [`volume create`] command.
|
|
|
|
The `volume create` command can trigger an expansion to occur, after the
|
|
volume has already been created or registered.
|
|
|
|
```shell-session
|
|
$ nomad volume create volume.hcl
|
|
Created host volume database with ID 0c903229-311d-ba8a-f77e-45c31b83fab3
|
|
```
|
|
|
|
1. Review the new capacity by running the [`volume status`] command.
|
|
|
|
```shell-session
|
|
$ nomad volume status -type=host 0c903229 | grep Capacity
|
|
Capacity = 100 GiB
|
|
```
|
|
|
|
[api_volume_create]: /nomad/api-docs/volumes#create-dynamic-host-volume
|
|
[api_volume_register]: /nomad/api-docs/volumes#register-dynamic-host-volume
|
|
[volume_source]: /nomad/docs/job-specification/volume#source
|
|
[capability]: /nomad/docs/other-specifications/volume/capability
|
|
[constraint]: /nomad/docs/job-specification/constraint
|
|
[`volume create`]: /nomad/commands/volume/create
|
|
[`volume register`]: /nomad/commands/volume/register
|
|
[`volume status`]: /nomad/commands/volume/status
|
|
[dhv_plugin]: /nomad/docs/architecture/storage/host-volumes
|
|
[mkdir_plugin]: #mkdir-plugin
|
|
[host_volumes_dir]: /nomad/docs/configuration/client#host_volumes_dir
|