nomad/website/content/plugins/drivers/exec2.mdx

---
layout: docs
page_title: Exec2 task driver plugin
description: >-
  The Exec2 task driver executes a command for a job task and leverages Linux kernel features such as Landlock LSM, cgroups v2, and the `unshare` system utility. Learn how to configure and install the Exec2 task driver plugin. Review capabilities, client requirements, and concepts such as filesystem and resource isolation. Learn how to use the Exec2 task driver in your Nomad job. Configure command, command args, paths, memory, CPU resources, and out-of-memory (OOM) behavior.
---

# Exec2 task driver plugin

Name: `exec2`

The `exec2` driver is used to execute a command for a task. It offers a security
model optimized for running 'ordinary' processes with very low startup times
and minimal overhead in terms of CPU, disk, and memory utilization. The `exec2`
driver leverages kernel features such as the [Landlock LSM][landlock], cgroups
v2, and the [`unshare`][unshare] system utility.

Review the source on [GitHub][github].

## Installation

<Tabs>
<Tab heading="Manual installation" group="manual">

You can download a [precompiled binary](https://releases.hashicorp.com/nomad-driver-exec2/) and verify the binary using the
available SHA-256 sums. After downloading nomad-driver-exec2 driver, unzip the package. Make sure
that the `nomad-driver-exec2` binary is available on your [plugin_dir](/nomad/docs/configuration#plugin_dir) path, specified by the client's config file, before continuing with the
other guides.

</Tab>
<Tab heading="Ubuntu/Debian" group="manual">

Install the required packages.

```shell-session
$ sudo apt-get update && \
  sudo apt-get install wget gpg coreutils
```

Add the HashiCorp [GPG key][gpg-key].

```shell-session
$ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
```

Add the official HashiCorp Linux test repository.

```shell-session
$ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) test" | sudo tee /etc/apt/sources.list.d/hashicorp.list
```

Update and install.

```shell-session
$ sudo apt-get update && sudo apt-get install nomad-driver-exec2
```

</Tab>
<Tab heading="CentOS/RHEL" group="linux">

Install `yum-config-manager` to manage your repositories.

```shell-session
$ sudo yum install -y yum-utils
```

Use `yum-config-manager` to add the official HashiCorp Linux repository.

```shell-session
$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
```

Edit the repo file at `/etc/yum.repos.d/hashicorp.repo` and set `enabled=1` for `[hashicorp-test]`.

Install.

```shell-session
$ sudo yum -y install nomad-driver-exec2
```

</Tab>
</Tabs>

## Usage

The example below is a short demonstration of what a task making use of the
exec2 task driver looks like:

```hcl
job "http" {
  group "web" {
    task "python" {
      driver = "exec2"
      config {
        command = "python3"
        args    = ["-m", "http.server", "8080", "--directory", "${NOMAD_TASK_DIR}"]
        unveil  = ["r:/etc/mime.types"]
      }
    }
  }
}
```

## Client Requirements

The `exec2` task driver is not built into Nomad. It must be [downloaded][releases]
onto the client host in the configured plugin directory.

- Linux 5.15+
- Cgroups v2 enabled
- Landlock LSM enabled
- Commands `nsenter` and `unshare`
- Nomad client running as root

## Capabilities

The `exec2` task driver implements the following driver [capabilities][capabilities]

| Feature              | Implementation    |
| -------------------- | ----------------- |
| `nomad alloc signal` | true              |
| `nomad alloc exec`   | false             |
| filesystem isolation | unveil            |
| network isolation    | host, group, none |
| volume mounting      | false             |

## Concepts

### Filesystem Isolation

##### landlock

The `exec2` task driver makes use of [`go-landlock`][landlock] for providing
filesystem isolation, making the host filesystem unreachable except where
explicitly allowed.

By default a task is enabled to access its task directory and its shared alloc
directory. The paths to these directories are accessible by reading the environment
variables `$NOMAD_TASK_DIR` and `$NOMAD_ALLOC_DIR` respectively.

A file access mode must also be specified when granting additional filesystem
access to a path. This is done by prefix the path with `'r'`, `'w'`, `'x'`,
and/or `'c'` indicating read, write, executable, and create permissions,
respectively. e.g.,

- <code>r:/srv/www</code> - read-only access to `/srv/www`
- <code>rwc:/tmp</code> - read, write, and create files in `/tmp`
- <code>rx:/opt/bin/application</code> - read and execute a specific application
- <code>wc:/var/log</code> - write and create files in `/var/log`

This style of permission control is modeled after the [`unveil`][unveil] system
call pioneered by the OpenBSD project. In configuration parameters we refer to
"unveil"-ing of filesystem paths as `exec2` is leveraging landlock to emulate
the semantics of `unveil`.

##### dynamic workload users

While landlock prevents tasks from accessing the host filesystem, Nomad 1.8
introduces `dynamic workload users` which enable tasks to be run as a UID/GID
that is not assigned to any user. This provides protection from non-root users
getting access inside the task and allocation directories created for the task.

To make use of a dynamic workload user, simply leave the `user` field blank in
the task definition of an `exec2` task.

### Resource Isolation

Similar to `exec` and other container runtimes, `exec2` makes use of cgroups for
limiting the amount of CPU and RAM a task may consume.

## Configuration

### Plugin Configuration

The default plugin configuration is shown below. System default paths are enabled,
but nothing else. These default paths enable basic functionality like reading
system TLS certificates, executing programs in `/bin`, `/usr/bin`, and accessing
system shared object files. The exact set of paths is system dependent, and can
be disabled or customized in plugin config.

The default set of paths are listed below. These paths are enabled only if they
are found to exist at the time of the task launching.

##### shared objects

```
rx:/lib
rx:/lib64
rx:/usr/lib
rx:/usr/libexec
rx:/usr/local/lib
rx:/usr/local/lib64
r:/etc/ld.so.conf
r:/etc/ld.so.cache
r:/etc/ld.so.conf.d
```

##### io, common

```
rwc:/tmp
rw:/dev/null
r:/dev/zero
r:/dev/fd
rw:/dev/stdin
rw:/dev/stdout
r:/dev/urandom
w:/dev/log
r:/usr/share/locale
r:/proc/self/cmdline
r:/usr/share/zoneinfo
r:/usr/share/common-licenses
r:/proc/sys/kernel/ngroups_max
r:/proc/sys/kernel/cap_last_cap
r:/proc/sys/vm/overcommit_memory
```

##### dns

```
r:/etc/hosts
r:/hostname
r:/etc/services
r:/etc/protocols
r:/etc/resolv.conf
```

##### certificates

```
r:/etc/ssl/certs
r:/etc/pki/tls/certs
r:/etc/ssl/ca-bundle.pem
r:/etc/pki/tls/cacert.pem
r:/etc/pki/ca-trust-extracted/pem/tls-ca-bundle.pem
r:/etc/ssl/cert.pem
```

Additional allowable paths can be specified at the plugin level, which applies
to all tasks making use of the `exec2` task driver, or at the task level, which
will apply specifically to each task.

```hcl
plugin "nomad-driver-exec2" {
  config {
    unveil_defaults = true
    unveil_paths    = []
    unveil_by_task  = false
  }
}
```

- `unveil_defaults` - (default: `true`) - enable or disable default system paths
  useful for running basic commands
- `unveil_paths` - (default: `[]`) - a list of filesystem paths with permissions
  to grant all tasks
  ```hcl
  unveil_paths = ["rx:/opt/bin", "r:/srv/certs"]
  ```
- `unveil_by_task` - (default: `false`) - enable or disable job submitters to
  specify additional filesystem path access within task config

### Task Configuration

##### config

Task configuration for an exec2 task includes setting a `command`, `args` for
the command, and additional `unveil` filesystem paths if `unveil_by_task` is
enabled in plugin configuration.

```hcl
config {
  command       = "/usr/bin/cat"
  args          = ["/etc/os-release"]
  unveil        = ["r:/etc/os-release"]
  oom_score_adj = 500
}
```

- `command` - (string: required) - command to run
- `args` - (list(string): optional) - list of arguments to provide
  to `command`
- `unveil` - (list(string): optional) - list of additional filesystem paths to
  provide access to the task (requires `unveil_by_task` in plugin config).
- `oom_score_adj` - (optional) - The likelihood of the task being OOM killed,
  must be a positive integer. Defaults to `0`.

##### cpu

Tasks are limited in CPU resources by setting the `cpu` or `cores` values in the
task `resources` block.

- `cpu` - (default: `100`) - limits the CPU bandwidth allowable for the task to
  make use of in MHz, may not be used with `cores`.
- `cores` - (int: optional) - specifies the number of CPU cores to reserve
  specifically for the task, may not be used with `cpu`

##### memory

Tasks are limited in memory resources by setting `memory` and optionally the
`memory_max` values in the task `resources` block.

### Node Attributes

When installed, the `exec2` plugin provides the following node attributes which
can be used as constraints when authoring jobs.

```
driver.exec2.unveil.defaults    = true
driver.exec2.unveil.tasks       = true
```

[capabilities]: /nomad/plugins/author/task-driver#task-driver-plugin-api
[github]: https://github.com/hashicorp/nomad-driver-exec2
[landlock]: https://github.com/shoenig/go-landlock
[releases]: https://releases.hashicorp.com/nomad-driver-exec2
[unshare]: https://www.man7.org/linux/man-pages/man1/unshare.1.html
[unveil]: https://man.openbsd.org/unveil