nomad/website/content/docs/configuration/server_join.mdx

---
layout: docs
page_title: server_join Block in Agent Configuration
description: >-
  Configure how Nomad agents discover and connect to Nomad servers in Amazon, Microsoft, and Google clouds as well as on-prem. Use the`server_join` block to configure the list of servers to join, retry interval, and max rejoin attempts.
---

# `server_join` Block in Agent Configuration

<Placement
  groups={[
    ['server', 'server_join'],
    ['client', 'server_join'],
  ]}
/>

This page provides reference information for configuring how Nomad agents
discover and connect to Nomad servers in Amazon, Microsoft, and Google
clouds as well as on-prem. Use the`server_join` block to configure the list of
servers to join, retry interval, and max rejoin attempts.

```hcl
server_join {
  retry_join     = [ "1.1.1.1", "2.2.2.2" ]
  retry_max      = 3
  retry_interval = "15s"
}
```

## `server_join` Parameters

- `retry_join` `(array<string>: [])` - Specifies a list of Nomad server
  addresses and [Cloud Auto-join](#cloud-auto-join) configurations that are
  joined as cluster members. This is similar to [`start_join`](#start_join),
  but join attempts are retried up to [retry_max](#retry_max) times.
  Further, `retry_join` is available to both Nomad servers and clients, while
  `start_join` is only defined for Nomad servers. This is useful for cases where
  we know the address will become available eventually. Use `retry_join` with an
  array as a replacement for `start_join`, **do not use both options**.

  Server addresses must conform to the [server address format](#server-address-format).

  ```hcl
  server_join {
    retry_join = [ "1.1.1.1", "2.2.2.2" ]
  }
  ```

  Auto-join configurations must conform to the [Cloud Auto-join format](#cloud-auto-join).

  Cloud-Auto join using `go-discover` to join an AWS EC2 cluster.

  ```hcl
  server_join {
    retry_join = [ "provider=aws tag_key=..." ]
  }
  ```

  Cloud-Auto join using `go-netaddrs` to join a Hetzner cluster of servers
  labeled with `nomad-server=true` using the `hcloud` CLI.

  ```hcl
  server_join {
    retry_join = [ "exec=hcloud server list -o noheader -o columns=ipv4 -l nomad-server=true | tr '\n' ' '" ]
  }
  ```

  Server addresses and Cloud Auto-join configurations may be used together.
  This is a mixed configuration containing a server address and Cloud Auto-join
  configuration.

  ```hcl
  server_join {
    retry_join = [ "1.1.1.1", "provider=aws tag_key=..." ]
  }
  ```

  Refer to [server address format](#server-address-format) for more information
  about expected server address formats and [Cloud Auto-join](#cloud-auto-join)
  for more information on expected Cloud Auto-join formats.

- `retry_interval` `(string: "30s")` - Specifies the time to wait between retry
  join attempts.

- `retry_max` `(int: 0)` - Specifies the maximum number of join attempts to be
  made before exiting with a return code of 1. By default, this is set to 0
  which is interpreted as infinite retries.

- `start_join` `(array<string>: [])` - Specifies a list of server addresses to
  join on startup. If Nomad is unable to join with any of the specified
  addresses, agent startup will fail. See the
  [server address format](#server-address-format) section for more information
  on the format of the string. This field is defined only for Nomad servers and
  will result in a configuration parse error if included in a client
  configuration.

## Server Address Format

This section describes the acceptable syntax and format for describing the
location of a Nomad server. There are many ways to reference a Nomad server,
including directly by IP address and resolving through DNS.

### Directly via IP Address

It is possible to address another Nomad server using its IP address. This is
done in the `ip:port` format, such as:

```
1.2.3.4:5678
```

If the IP address is an IPv6 address, it must be in URL format surrounded by
brackets. For example:

```
[2001:db8::1]:5678
```

If the port option is omitted, it defaults to the Serf port, which is 4648
unless configured otherwise:

```
1.2.3.4 => 1.2.3.4:4648
```

### Via Domains or DNS

It is possible to address another Nomad server using its DNS address. This is
done in the `address:port` format, such as:

```
nomad-01.company.local:5678
```

If the port option is omitted, it defaults to the Serf port, which is 4648
unless configured otherwise:

```
nomad-01.company.local => nomad-01.company.local:4648
```

### Via the go-discover interface
`retry_join` accepts a unified interface using the
[go-discover](https://github.com/hashicorp/go-discover) library for doing
automated cluster joining using cloud provider metadata. Refer to [Cloud
Auto-join](#cloud-auto-join) for more information.

```
"provider=aws tag_key=..." => 1.2.3.4:4648
```

## Cloud Auto-join

`retry_join`'s Cloud Auto-join allows Nomad to automatically discover cluster
server addresses using cloud provider metadata. Cloud Auto-join allows both
[go-discover](https://github.com/hashicorp/go-discover) and
[go-netaddrs](https://github.com/hashicorp/go-netaddrs) formats.

Configurations prefixed with `provider=` use `go-discover` whereas
configurations prefixed with `exec=` use `go-netaddrs`. For cloud providers not
supported by `go-discover`, use `go-netaddrs`.

### `go-discover` Configurations

`go-discover` configurations are prefixed with `provider=`.

The following sections describe the Cloud Auto-join `retry_join` options that are specific
to a subset of supported cloud providers. For information on all providers,
refer to further
documentation in [go-discover](https://github.com/hashicorp/go-discover).

#### Amazon EC2

This returns the first private IP address of all servers in the given
region which have the given `tag_key` and `tag_value`.

```json
{
  "retry_join": ["provider=aws tag_key=... tag_value=..."]
}
```

- `provider` (required) - the name of the provider ("aws" in this case).
- `tag_key` (required) - the key of the tag to auto-join on.
- `tag_value` (required) - the value of the tag to auto-join on.
- `region` (optional) - the AWS region to authenticate in.
- `addr_type` (optional) - the type of address to discover: `private_v4`, `public_v4`, `public_v6`. Default is `private_v4`. (>= 1.0)
- `access_key_id` (optional) - the AWS access key for authentication. Refer to
  the following Authentication and Precedence section for details.
- `secret_access_key` (optional) - the AWS secret access key for authentication.
 Refer to the following Authentication and Precedence section for details.

##### Authentication and Precedence

- Static credentials `access_key_id=... secret_access_key=...`
- Environment variables (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`)
- Shared credentials file (`~/.aws/credentials` or the path specified by `AWS_SHARED_CREDENTIALS_FILE`)
- ECS task role metadata (container-specific).
- EC2 instance role metadata.

  The only required IAM permission is `ec2:DescribeInstances`, and it is
  recommended that you make a dedicated key used only for auto-joining. If the
  region is omitted it will be discovered through the local instance's [EC2
  metadata
  endpoint](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-identity-documents.html).

#### Microsoft Azure

This returns the first private IP address of all servers in the given region
which have the given `tag_key` and `tag_value` in the tenant and subscription, or in
the given `resource_group` of a `vm_scale_set` for Virtual Machine Scale Sets.

```json
{
  "retry_join": [
    "provider=azure tag_name=... tag_value=... tenant_id=... client_id=... subscription_id=... secret_access_key=..."
  ]
}
```

- `provider` (required) - the name of the provider ("azure" in this case).
- `tenant_id` (required) - the tenant to join machines in.
- `client_id` (required) - the client to authenticate with.
- `secret_access_key` (required) - the secret client key.

Use these configuration parameters when using tags:

- `tag_name` - the name of the tag to auto-join on.
- `tag_value` - the value of the tag to auto-join on.

Use these configuration parameters when using Virtual Machine Scale Sets (Consul 1.0.3 and later):

- `resource_group` - the name of the resource group to filter on.
- `vm_scale_set` - the name of the virtual machine scale set to filter on.

  When using tags the only permission needed is the `ListAll` method for `NetworkInterfaces`. When using
  Virtual Machine Scale Sets the only role action needed is `Microsoft.Compute/virtualMachineScaleSets/*/read`.

#### Google Compute Engine

This returns the first private IP address of all servers in the given
project which have the given `tag_value`.

```json
{
  "retry_join": ["provider=gce project_name=... tag_value=..."]
}
```

- `provider` (required) - the name of the provider ("gce" in this case).
- `tag_value` (required) - the value of the tag to auto-join on.
- `project_name` (optional) - the name of the project to auto-join on. Discovered if not set.
- `zone_pattern` (optional) - the list of zones can be restricted through an RE2 compatible regular expression. If omitted, servers in all zones are returned.
- `credentials_file` (optional) - the credentials file for authentication. Refer
 to the following Authentication and Precedence section for more information.

##### Authentication and Precedence

- Use credentials from `credentials_file`, if provided.
- Use JSON file from `GOOGLE_APPLICATION_CREDENTIALS` environment variable.
- Use JSON file in a location known to the gcloud command-line tool.
- On Windows, this is `%APPDATA%/gcloud/application_default_credentials.json`.
- On other systems, `$HOME/.config/gcloud/application_default_credentials.json`.
- On Google Compute Engine, use credentials from the metadata
  server. In this final case any provided scopes are ignored.

Discovery requires a [GCE Service
Account](https://cloud.google.com/compute/docs/access/service-accounts).
Credentials are searched using the following paths, in order of precedence.


### `go-netaddrs` Configurations

`go-netaddrs` configurations are prefixed with `exec=`.

What follows the `exec=` prefix may be any executable program and its arguments.
Commands run by `go-netaddrs` must return a list of space-delimited IPv4 or IPv6
addresses and exit with code `0` on success and non-zero on failure.

You can refer to the executable by its absolute file system path, or by
the executable's name if it can be found on the Nomad agent's PATH. Refer to
[executables in the current directory](https://pkg.go.dev/os/exec#hdr-Executables_in_the_current_directory)
for more details on the lookup behavior.

Example `go-netaddrs` configuration.

```hcl
server_join {
  retry_join = [ "exec=hcloud server list -o noheader -o columns=ipv4 -l nomad-server=true | tr '\n' ' '" ]
}
```

Here, `hcloud`'s newline-delimited output has been re-formatted as space-
delimited by piping its output into `tr`.

Output of an example `go-netaddrs` executable

```
"1.1.1.1 2.2.2.2"
```

Visit [go-netaddrs](https://github.com/hashicorp/go-netaddrs) for more
information on go-netaddrs configuration.