mirror of
https://github.com/kemko/nomad.git
synced 2026-01-04 01:15:43 +03:00
986f3c727a
* action page * change all page_title fields * update title * constraint through migrate pages * update page title and heading to use sentence case * fix front matter description * Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
101 lines
3.4 KiB
Plaintext
101 lines
3.4 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: numa block in the job specification
|
|
description: |-
|
|
Define a NUMA-aware task scheduling strategy in the `numa` block of the Nomad job specification. Configure the affinity strategy and specify a list of devices that must be colocated on the same NUMA node.
|
|
---
|
|
|
|
# `numa` block in the job specification
|
|
|
|
<Placement groups={['job', 'group', 'task', 'resources', 'numa']} />
|
|
|
|
The `numa` block is used to configure how Nomad will assign CPU cores for a task
|
|
while taking the [NUMA hardware topology][numa_wiki] of a node into consideration.
|
|
Workloads that are sensitive to memory latency can perform significantly better
|
|
when pinned to CPU cores on the same NUMA node.
|
|
|
|
NUMA aware scheduling is currently limited to Linux.
|
|
|
|
<EnterpriseAlert product="nomad"/>
|
|
|
|
```hcl
|
|
job "example" {
|
|
group "group" {
|
|
task "task" {
|
|
resources {
|
|
cores = 8
|
|
numa {
|
|
affinity = "require"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
In the example above, the task is requesting Nomad to reserve 8 CPU cores, all
|
|
of which must be colocated on the same hardware NUMA node. The Nomad scheduler
|
|
will ensure the task is placed on a Nomad node with sufficient unused CPU cores
|
|
in a compatible configuration.
|
|
|
|
Configuring the `numa` block requires the task specifies CPU resources using
|
|
the [`cores`][cores] parameter.
|
|
|
|
# Parameters
|
|
|
|
- `affinity` `(string: "none")` - Specifies the strategy Nomad will use when
|
|
selecting CPU cores to assign to a task. Possible values are `"none"`,
|
|
`"prefer"`, or `"require"`.
|
|
- `none` - Nomad is free to allocate CPU cores using any strategy. Nomad uses
|
|
this freedom to allocate cores in such a way that minimizes the amount of
|
|
fragmentation of core availability per NUMA node. It does so by bin-packing
|
|
the chosen cores onto the NUMA nodes with the fewest number of cores available.
|
|
- `prefer` - Nomad will select the set of CPU cores on a node that minimizes
|
|
the total distance between those cores, but does not limit those CPU core
|
|
selections to come from a single NUMA node.
|
|
- `require` - Nomad will select a set of CPU cores that are strictly colocated
|
|
on the same hardware NUMA node. If there are no Nomad nodes with a sufficient
|
|
number of available cores in a compatible configuration, task placement will
|
|
fail due to exhausted resources.
|
|
|
|
- `devices` `(list<string>: nil)` - Specifies which devices must be colocated on
|
|
the name NUMA node, along with allocated CPU cores. Must be a subset of the
|
|
devices listed in the [resources] block. May only be used with `affinity` set
|
|
to `require`.
|
|
|
|
<Note>
|
|
|
|
The `require` affinity option causes fragmentation of available CPU cores
|
|
based on the NUMA node the selected cores are associated with. Use it for
|
|
workloads known to be highly sensitive to memory latencies. A cluster where
|
|
all jobs make use of the `require` affinity option will not be able to make
|
|
efficient use of available resources.
|
|
</Note>
|
|
|
|
## Examples
|
|
|
|
This example will allocate a `1080ti` GPU and ensure it is on the same NUMA node
|
|
as the 4 CPU cores reserved for the task.
|
|
|
|
```hcl
|
|
resources {
|
|
cores = 4
|
|
memory = 2048
|
|
|
|
device "nvidia/gpu/1080ti" {
|
|
count = 1
|
|
}
|
|
|
|
numa {
|
|
affinity = "require"
|
|
devices = [
|
|
"nvidia/gpu/1080ti"
|
|
]
|
|
}
|
|
}
|
|
```
|
|
|
|
[numa_wiki]: https://en.wikipedia.org/wiki/Non-uniform_memory_access
|
|
[cores]: /nomad/docs/job-specification/resources#cores
|
|
[resources]: /nomad/docs/job-specification/resources
|