nomad/website/content/docs/drivers/exec.mdx

---
layout: docs
page_title: 'Drivers: Exec'
description: The Exec task driver runs binaries using OS isolation primitives.
---

# Isolated Fork/Exec Driver

Name: `exec`

The `exec` driver is used to simply execute a particular command for a task.
However, unlike [`raw_exec`](/nomad/docs/drivers/raw_exec) it uses the underlying isolation
primitives of the operating system to limit the task's access to resources. While
simple, since the `exec` driver can invoke any command, it can be used to call
scripts or other wrappers which provide higher level features.

## Task Configuration

```hcl
task "webservice" {
  driver = "exec"

  config {
    command = "my-binary"
    args    = ["-flag", "1"]
  }
}
```

The `exec` driver supports the following configuration in the job spec:

- `command` - The command to execute. Must be provided. If executing a binary
  that exists on the host, the path must be absolute and within the task's
  [chroot](#chroot) or in a [host volume][] mounted with a
  [`volume_mount`][volume_mount] block. The driver will make the binary
  executable and will search, in order:

  - The `local` directory with the task directory.
  - The task directory.
  - Any mounts, in the order listed in the job specification.
  - The `usr/local/bin`, `usr/bin` and `bin` directories inside the task
    directory.

  If executing a binary that is downloaded
  from an [`artifact`](/nomad/docs/job-specification/artifact), the path can be
  relative from the allocation's root directory.

- `args` - (Optional) A list of arguments to the `command`. References
  to environment variables or any [interpretable Nomad
  variables](/nomad/docs/runtime/interpolation) will be interpreted before
  launching the task.

- `pid_mode` - (Optional) Set to `"private"` to enable PID namespace isolation for
  this task, or `"host"` to disable isolation. If left unset, the behavior is
  determined from the [`default_pid_mode`][default_pid_mode] in plugin configuration.

!> **Warning:** If set to `"host"`, other processes running as the same user will
be able to access sensitive process information like environment variables.

- `ipc_mode` - (Optional) Set to `"private"` to enable IPC namespace isolation for
  this task, or `"host"` to disable isolation. If left unset, the behavior is
  determined from the [`default_ipc_mode`][default_ipc_mode] in plugin configuration.

!> **Warning:** If set to `"host"`, other processes running as the same user will be
able to make use of IPC features, like sending unexpected POSIX signals.

- `cap_add` - (Optional) A list of Linux capabilities to enable for the task.
  Effective capabilities (computed from `cap_add` and `cap_drop`) must be a
  subset of the allowed capabilities configured with [`allow_caps`][allow_caps].
  Note that `"all"` is not permitted here if the `allow_caps` field in the
  driver configuration doesn't also allow all capabilities.

```hcl
config {
  cap_add = ["net_raw", "sys_time"]
}
```

- `cap_drop` - (Optional) A list of Linux capabilities to disable for the task.
  Effective capabilities (computed from `cap_add` and `cap_drop`) must be a subset
  of the allowed capabilities configured with [`allow_caps`][allow_caps].

```hcl
config {
  cap_drop = ["all"]
  cap_add  = ["chown", "sys_chroot", "mknod"]
}
```

- `work_dir` - (Optional) Sets a custom working directory for the task. This path must be
  absolute and within the task's [chroot](#chroot) or in a [host volume][] mounted
  with a [`volume_mount`][volume_mount] block. This will also change the working
  directory when using `nomad alloc exec`.

## Examples

To run a binary present on the Node:

```hcl
task "example" {
  driver = "exec"

  config {
    # When running a binary that exists on the host, the path must be absolute.
    command = "/bin/sleep"
    args    = ["1"]
  }
}
```

To execute a binary downloaded from an
[`artifact`](/nomad/docs/job-specification/artifact):

```hcl
task "example" {
  driver = "exec"

  config {
    command = "name-of-my-binary"
  }

  artifact {
    source = "https://internal.file.server/name-of-my-binary"
    options {
      checksum = "sha256:abd123445ds4555555555"
    }
  }
}
```

## Capabilities

The `exec` driver implements the following [capabilities](/nomad/docs/concepts/plugins/task-drivers#capabilities-capabilities-error).

| Feature              | Implementation |
| -------------------- | -------------- |
| `nomad alloc signal` | true           |
| `nomad alloc exec`   | true           |
| filesystem isolation | chroot         |
| network isolation    | host, group    |
| volume mounting      | all            |

## Client Requirements

The `exec` driver can only be run when on Linux and running Nomad as root.
`exec` is limited to this configuration because currently isolation of resources
is only guaranteed on Linux. Further, the host must have cgroups mounted properly
in order for the driver to work.

If you are receiving the error:

```
* Constraint "missing drivers" filtered <> nodes
```

and using the exec driver, check to ensure that you are running Nomad as root.
This also applies for running Nomad in -dev mode.

## Plugin Options

- `default_pid_mode` `(string: optional)` - Defaults to `"private"`. Set to
  `"private"` to enable PID namespace isolation for tasks by default, or `"host"` to
  disable isolation.

!> **Warning:** If set to `"host"`, other processes running as the same user will
be able to access sensitive process information like environment variables.

- `default_ipc_mode` `(string: optional)` - Defaults to `"private"`. Set to
  `"private"` to enable IPC namespace isolation for tasks by default,
  or `"host"` to disable isolation.

!> **Warning:** If set to `"host"`, other processes running as the same user will be
able to make use of IPC features, like sending unexpected POSIX signals.

- `no_pivot_root` `(bool: optional)` - Defaults to `false`. When `true`, the driver uses `chroot`
  for file system isolation without `pivot_root`. This is useful for systems
  where the root is on a ramdisk.

- `allow_caps` - A list of allowed Linux capabilities. Defaults to

```hcl
["audit_write", "chown", "dac_override", "fowner", "fsetid", "kill", "mknod",
 "net_bind_service", "setfcap", "setgid", "setpcap", "setuid", "sys_chroot"]
```

  which is modeled after the capabilities allowed by [docker by default][docker_caps]
  (without [`NET_RAW`][no_net_raw]). Allows the operator to control which capabilities
  can be obtained by tasks using [`cap_add`][cap_add] and [`cap_drop`][cap_drop] options.
  Supports the value `"all"` as a shortcut for allow-listing all capabilities supported
  by the operating system.

!> **Warning:** Allowing more capabilities beyond the default may lead to
undesirable consequences, including untrusted tasks being able to compromise the
host system.

- `denied_host_uids` - (Optional) Specifies a comma-separated list of host uids to
  deny. Ranges can be specified by using a hyphen separating the two inclusive ends.
  If a "user" value is specified in task configuration and that user has a user id in
  the given ranges, the task will error before starting. This will not be checked on Windows
  clients.

```hcl
config {
  denied_host_uids = "0,10-15,22"
}
```

- `denied_host_gids` - (Optional) Specifies a comma-separated list of host gids to
  deny. Ranges can be specified by using a hyphen separating the two inclusive ends.
  If a "user" value is specified in task configuration and that user is part of
  any groups with gid's in the specified ranges, the task will error before
  starting. This will not be checked on Windows clients.

```hcl
config {
  denied_host_gids = "2,4-8"
}
```

## Client Attributes

The `exec` driver will set the following client attributes:

- `driver.exec` - This will be set to "1", indicating the driver is available.

## Resource Isolation

The resource isolation provided varies by the operating system of
the client and the configuration.

On Linux, Nomad will use cgroups, and a chroot to isolate the resources of a
process and as such the Nomad agent must be run as root. Some Linux
distributions do not boot with all required cgroups enabled by default. You
can see which cgroups are enabled by reading `/proc/cgroups`, and verifying
that all the following cgroups are enabled:

```
$ awk '{print $1 " " $4}' /proc/cgroups
#subsys_name enabled
cpuset 1
cpu 1
cpuacct 1
blkio 1
memory 1
devices 1
freezer 1
net_cls 1
perf_event 1
net_prio 1
hugetlb 1
pids 1
```

Nomad can only use cgroups to control resources if all the required controllers
are available. If one or more required cgroups are unavailable, Nomad will
disable resource controls that require cgroups entirely. See the documentation
on [cgroup controller requirements][] for more details.

### Chroot

Chroot changes the root directory of the current process and all its child processes. Nomad uses a chroot environment to implement file system isolation.

Nomad populates the chroot environment with data in the following host machine directories:

```
[
  "/bin",
  "/etc",
  "/lib",
  "/lib32",
  "/lib64",
  "/run/resolvconf",
  "/sbin",
  "/usr",
]
```

Nomad populates the task's chroot environment by linking or copying the data from the host into the chroot. Note that this can take considerable disk space.
The client manages garbage collection locally, which mitigates any issue this may create.

@include 'chroot-limitations.mdx'

Configure the chroot environment list through the agent client's
[`chroot_env` attribute](/nomad/docs/configuration/client#chroot_env).

### CPU

Nomad limits exec tasks' CPU based on CPU shares. CPU shares allow containers to
burst past their CPU limits. CPU limits will only be imposed when there is contention
for resources. When the host is under load your process may be throttled to stabilize
QoS depending on how many shares it has. You can see how many CPU shares are available
to your process by reading [`NOMAD_CPU_LIMIT`][runtime_env]. 1000 shares are approximately equal to 1 GHz.

Please keep the implications of CPU shares in mind when you load test workloads on Nomad.

If resources [`cores`][cores] is set, the task is given an isolated reserved set of
CPU cores to make use of. The total set of cores the task may run on is the private
set combined with the variable set of unreserved cores. The private set of CPU cores
is available to your process by reading [`NOMAD_CPU_CORES`][runtime_env].

[default_pid_mode]: /nomad/docs/drivers/exec#default_pid_mode
[default_ipc_mode]: /nomad/docs/drivers/exec#default_ipc_mode
[cap_add]: /nomad/docs/drivers/exec#cap_add
[cap_drop]: /nomad/docs/drivers/exec#cap_drop
[no_net_raw]: /nomad/docs/upgrade/upgrade-specific#nomad-1-1-0-rc1-1-0-5-0-12-12
[allow_caps]: /nomad/docs/drivers/exec#allow_caps
[docker_caps]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
[host volume]: /nomad/docs/configuration/client#host_volume-block
[volume_mount]: /nomad/docs/job-specification/volume_mount
[cores]: /nomad/docs/job-specification/resources#cores
[runtime_env]: /nomad/docs/runtime/environment#job-related-variables
[cgroup controller requirements]: /nomad/docs/install/production/requirements#hardening-nomad