diff --git a/demo/vagrant/Vagrantfile b/demo/vagrant/Vagrantfile index 1742b1ee1..34d0448f6 100644 --- a/demo/vagrant/Vagrantfile +++ b/demo/vagrant/Vagrantfile @@ -7,8 +7,7 @@ sudo apt-get update sudo DEBIAN_FRONTEND=noninteractive apt-get install -y unzip curl vim \ apt-transport-https \ ca-certificates \ - software-properties-common \ - openssl + software-properties-common # Download Nomad NOMAD_VERSION=0.6.0 @@ -68,6 +67,13 @@ EOF sudo systemctl enable consul.service sudo systemctl start consul +for bin in cfssl cfssl-certinfo cfssljson +do + echo "Installing $bin..." + curl -sSL https://pkg.cfssl.org/R1.2/${bin}_linux-amd64 > /tmp/${bin} + sudo install /tmp/${bin} /usr/local/bin/${bin} +done + SCRIPT Vagrant.configure(2) do |config| diff --git a/website/source/guides/tls.html.md b/website/source/guides/tls.html.md index 98f1a8e07..ba325e735 100644 --- a/website/source/guides/tls.html.md +++ b/website/source/guides/tls.html.md @@ -24,7 +24,7 @@ the expected region and configured for the expected role (e.g. `client.us-west.nomad`). Configuring TLS can be unfortunately complex process, but if you used the -[Getting Started guide's Vagrantfile][Vagrantfile] or have [OpenSSL][] and Nomad +[Getting Started guide's Vagrantfile][Vagrantfile] or have [cfssl][] and Nomad installed this guide will provide you with a production ready TLS configuration. @@ -38,72 +38,75 @@ XXX TODO XXX - serf encryption key The first step to configuring TLS for Nomad is generating certificates. In order to prevent unauthorized cluster access, Nomad requires all certificates -are signed by the sign Certificate Authority (CA). This should be a *private* -CA and not a public like [Let's Encrypt][letsencrypt] as any certificate signed -by this CA will be allowed to communicate with the cluster. +be signed by the same Certificate Authority (CA). This should be a *private* CA +and not a public one like [Let's Encrypt][letsencrypt] as any certificate +signed by this CA will be allowed to communicate with the cluster. ### Certificate Authority -You can generate a private CA certificate and key with OpenSSL: +There are a variety of tools for managing your own CA, [like the PKI secret +backend in Vault][vault-pki], but for the sake of simplicity in this guide +we'll use [cfssl][]. You can generate a private CA certificate and key with +[cfssl][]: ```shell -# Generate the CA's private key -# This file (nomad-ca.key) must be kept *secret* -openssl genrsa -out nomad-ca.key 4096 - -# Generate the CA's self-signed certicate -# This file (nomad-ca.crt) will be distributed to all nodes -openssl req -new -x509 -key nomad-ca.key -out nomad-ca.crt -You are about to be asked to enter information that will be incorporated -into your certificate request. -What you are about to enter is what is called a Distinguished Name or a DN. -There are quite a few fields but you can leave some blank -For some fields there will be a default value, -If you enter '.', the field will be left blank. ------ -Country Name (2 letter code) [AU]:. -State or Province Name (full name) [Some-State]:. -Locality Name (eg, city) []:. -Organization Name (eg, company) [Internet Widgits Pty Ltd]:. -Organizational Unit Name (eg, section) []:. -Common Name (e.g. server FQDN or YOUR name) []:Nomad CA -Email Address []:. +# Generate the CA's private key and certificate +cfssl print-defaults csr | cfssl gencert -initca - | cfssljson -bare nomad-ca ``` -Your answers to OpenSSL's prompts are purely informational and not used by -Nomad. - -The CA key (`nomad-ca.key`) will be used to sign certificates for Nomad nodes -and must be kept private. The CA certificate (`nomad-ca.crt`) contains the -public key necessary to validate Nomad certificates and therefore must be +The CA key (`nomad-ca-key.pem`) will be used to sign certificates for Nomad +nodes and must be kept private. The CA certificate (`nomad-ca.pem`) contains +the public key necessary to validate Nomad certificates and therefore must be distributed to every node that requires access. ### Node Certificates Once you have a CA certifacte and key you can generate and sign the -certificates Nomad will use directly. Traditionally TLS certificates use the +certificates Nomad will use directly. TLS certificates commonly use the fully-qualified domain name of the system being identified as the certificate's Common Name (CN). However, hosts (and therefore hostnames and IPs) are often ephemeral in Nomad clusters. They come and go as clusters are scaled up and down or outages occur. Not only would signing a new certificate per Nomad node be difficult, but using a hostname provides no security or functional benefits -to Nomad. To fulfill the desired security properties (see above) Nomad -certificates are signed with their region and role such as: +to Nomad. To fulfill the desired security properties (above) Nomad certificates +are signed with their region and role such as: * `client.global.nomad` for a client node in the `global` region * `server.us-west.nomad` for a server node in the `us-west` region To create certificates for the client and server in the cluster from the -[Getting Started guide][guide-cluster] with OpenSSL create the following -configuration file `nomad.conf`: +[Getting Started guide][guide-cluster] with [cfssl][] create ([or +download][cfssl.json]) the following configuration file as `cfssl.json` to +increase the default certificate expiration time: -```ini -basicConstraints = CA:FALSE -subjectAltName = @alt_names +```json +{ + "signing": { + "default": { + "expiry": "87600h", + "usages": [ + "signing", + "key encipherment", + "server auth", + "client auth" + ] + } + } +} +``` -[alt_names] -DNS.1 = ${commonName} -DNS.2 = localhost +```shell +# Generate a certificate for the Nomad server +echo '{}' | cfssl gencert -ca=nomad-ca.pem -ca-key=nomad-ca-key.pem -config=cfssl.json \ + -hostname="server.global.nomad,localhost" - | cfssljson -bare server + +# Generate a certificate for the Nomad client +echo '{}' | cfssl gencert -ca=nomad-ca.pem -ca-key=nomad-ca-key.pem -config=cfssl.json \ + -hostname="client.global.nomad,localhost" - | cfssljson -bare client + +# Generate a certificate for the CLI +echo '{}' | cfssl gencert -ca nomad-ca.pem -ca-key nomad-ca-key.pem -profile=client \ + - | cfssljson -bare cli ``` Using `localhost` as a subject alternate name (SAN) allows tools like `curl` to @@ -111,61 +114,32 @@ be able to communicate with Nomad's HTTP API when run on the same host. Other SANs may be added including a DNS resolvable hostname to allow remote HTTP requests from third party tools. -Then create client and server certificate and key pairs: - -```shell -# Client key and certificate -openssl genrsa -out client.global.nomad.key 4096 -openssl req -new -sha256 \ - -out client.global.nomad.csr \ - -key client.global.nomad.key \ - -subj /CN=client.global.nomad/ -openssl x509 -req \ - -in client.global.nomad.csr \ - -CA nomad-ca.crt \ - -CAkey nomad-ca.key \ - -days 3650 \ - -set_serial $(hexdump -e '"0x%x%x%x%x"' -n 16 /dev/urandom) \ - -extfile nomad.conf \ - -out client.global.nomad.crt - -# Server key and certificate -openssl genrsa -out server.global.nomad.key 4096 -openssl req -new -sha256 \ - -out server.global.nomad.csr \ - -key server.global.nomad.key \ - -subj /CN=server.global.nomad/ -openssl x509 -req \ - -in server.global.nomad.csr \ - -CA nomad-ca.crt \ - -CAkey nomad-ca.key \ - -days 3650 \ - -set_serial $(hexdump -e '"0x%x%x%x%x"' -n 16 /dev/urandom) \ - -extfile nomad.conf \ - -out server.global.nomad.crt -``` - You should now have the following files: -* `nomad-ca.key` - CA private key. Keep safe! -* `nomad-ca.crt` - CA public certificate. -* `client.global.nomad.key` - Nomad client node private key for the `global` region. -* `client.global.nomad.csr` - Nomad client node certificate signing request for the `global` region. -* `client.global.nomad.crt` - Nomad client node public certificate for the `global` region. -* `server.global.nomad.key` - Nomad server node private key for the `global` region. -* `server.global.nomad.csr` - Nomad server node certificate signing request for the `global` region. -* `server.global.nomad.crt` - Nomad server node public certificate for the `global` region. +* `cfssl.json` - cfssl configuration. +* `nomad-ca.csr` - CA signing request. +* `nomad-ca-key.pem` - CA private key. Keep safe! +* `nomad-ca.pem` - CA public certificate. +* `cli.csr` - Nomad CLI certificate signing request. +* `cli.pem` - Nomad CLI certificate. +* `cli-key.pem` - Nomad CLI private key. +* `client.csr` - Nomad client node certificate signing request for the `global` region. +* `client-key.pem` - Nomad client node private key for the `global` region. +* `client.pem` - Nomad client node public certificate for the `global` region. +* `server.csr` - Nomad server node certificate signing request for the `global` region. +* `server-key.pem` - Nomad server node private key for the `global` region. +* `server.pem` - Nomad server node public certificate for the `global` region. -Each Nomad node should have the appropriate key (`.key`) and certificate -(`.crt`) file for its region and role. In addition each node needs the CA's -public certificate (`nomad-ca.crt`). +Each Nomad node should have the appropriate key (`-key.pem`) and certificate +(`.pem`) file for its region and role. In addition each node needs the CA's +public certificate (`nomad-ca.pem`). ## Configuring Nomad Once you have the appropriate key and certificates installed you're ready to configure Nomad to use them for mTLS. Starting with the [server configuration -from the Getting Started guide][guide-server] add the following TLS specific -configuration options: +from the Getting Started guide][guide-server] add the following TLS +CONFIGUration options: ```hcl # Increase log verbosity @@ -187,9 +161,9 @@ tls { http = true rpc = true - ca_file = "nomad-ca.crt" - cert_file = "server.global.nomad.crt" - key_file = "server.global.nomad.key" + ca_file = "nomad-ca.pem" + cert_file = "server.pem" + key_file = "server-key.pem" verify_server_hostname = true verify_https_client = true @@ -208,13 +182,13 @@ doesn't use separate ports for TLS and non-TLS traffic: your cluster should either use TLS or not. ```hcl - ca_file = "nomad-ca.crt" - cert_file = "server.global.nomad.crt" - key_file = "server.global.nomad.key" + ca_file = "nomad-ca.pem" + cert_file = "server.pem" + key_file = "server-key.pem" ``` The file lines should point to whereever you placed the certificate files on -the node. This guide assumes they're in Nomad's current directory. +the node. This guide assumes they are in Nomad's current directory. ```hcl verify_server_hostname = true @@ -227,11 +201,12 @@ cerificate will be checked to ensure it is signed by the same CA, but its role and region will not be verified. This means any service with a certificate from the same CA as Nomad can act as a client or server of any region. -`verify_https_client` may be disabled to allow HTTP API clients (eg Nomad CLI, Consul, or -curl) to communicate with the HTTPS API without presenting a client-side -certificate. If `verify_https_client` is enabled ony HTTP API clients -presenting a certificate signed by the same CA as Nomad's certificate are -allowed to access Nomad. +`verify_https_client` requires HTTP API clients to present a certificate signed +by the same CA as Nomad's certificate. It may be disabled to allow HTTP API +clients (eg Nomad CLI, Consul, or curl) to communicate with the HTTPS API +without presenting a client-side certificate. If `verify_https_client` is +enabled ony HTTP API clients presenting a certificate signed by the same CA as +Nomad's certificate are allowed to access Nomad. ~> Enabling `verify_https_client` feature effectively protects Nomad from unauthorized network access at the cost of breaking compatibility with Consul @@ -269,9 +244,9 @@ tls { http = true rpc = true - ca_file = "nomad-ca.crt" - cert_file = "client.global.nomad.crt" - key_file = "client.global.nomad.key" + ca_file = "nomad-ca.pem" + cert_file = "client.pem" + key_file = "client-key.pem" verify_server_hostname = true verify_https_client = true @@ -307,17 +282,33 @@ Don't worry, the Nomad CLI just defaults to `http://...` instead of ```shell export NOMAD_ADDR=https://localhost:4646 -export NOMAD_CACERT=nomad-ca.crt -export NOMAD_CLIENT_CERT=client.global.nomad.crt +export NOMAD_CACERT=nomad-ca.pem +export NOMAD_CLIENT_CERT=client.pem +export NOMAD_CLIENT_KEY=client-key.pem ``` The `NOMAD_CACERT` also needs to be set so the CLI can verify it's talking to -an actual Nomad node. Finally, the `NOMAD_CLIENT_CERT` needs to be set since we -enabled `verify_https_client` above which prevents any access lacking a client -certificate. Operators may wish to generate a certificate specifically for the -CLI as any certificate signed by Nomad's CA will work. +an actual Nomad node. Finally, `NOMAD_CLIENT_CERT` and `NOMAD_CLIENT_KEY` need +to be set since we enabled `verify_https_client` above which prevents any +access lacking a client certificate. -XXX TODO XXX - an example of everything working +Now the CLI works as expected: + +```text +vagrant@nomad:~$ nomad node-status +ID DC Name Class Drain Status +237cd4c5 dc1 nomad false ready + +vagrant@nomad:~$ nomad init +Example job file written to example.nomad +vagrant@nomad:~$ nomad run example.nomad +==> Monitoring evaluation "e9970e1d" + Evaluation triggered by job "example" + Allocation "a1f6c3e7" created: node "237cd4c5", group "cache" + Evaluation within deployment: "080460ce" + Evaluation status changed: "pending" -> "complete" +==> Evaluation "e9970e1d" finished with status "complete" +``` ## Switching an existing cluster to TLS @@ -326,6 +317,8 @@ XXX TODO XXX [guide-server]: https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/server.hcl [guide-cluster]: https://www.nomadproject.io/intro/getting-started/cluster.html [letsencrypt]: https://letsencrypt.org/ -[OpenSSL]: https://www.openssl.org/ +[cfssl]: https://cfssl.org/ +[cfssl.json]: https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/cfssl.json [Vagrantfile]: https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/Vagrantfile -[Vault]: https://www.vaultproject.io/docs/secrets/pki/index.html +[Vault]: https://www.vaultproject.io/ +[vault-pki]: https://www.vaultproject.io/docs/secrets/pki/index.html