diff --git a/website/source/assets/images/eval-flow.png b/website/source/assets/images/eval-flow.png deleted file mode 100644 index 193a22898..000000000 Binary files a/website/source/assets/images/eval-flow.png and /dev/null differ diff --git a/website/source/assets/images/global-arch.png b/website/source/assets/images/global-arch.png deleted file mode 100644 index c5fa23881..000000000 Binary files a/website/source/assets/images/global-arch.png and /dev/null differ diff --git a/website/source/assets/images/nomad-architecture-global.png b/website/source/assets/images/nomad-architecture-global.png new file mode 100644 index 000000000..62adc2fc2 Binary files /dev/null and b/website/source/assets/images/nomad-architecture-global.png differ diff --git a/website/source/assets/images/nomad-architecture-region.png b/website/source/assets/images/nomad-architecture-region.png new file mode 100644 index 000000000..3d563b674 Binary files /dev/null and b/website/source/assets/images/nomad-architecture-region.png differ diff --git a/website/source/assets/images/nomad-data-model.png b/website/source/assets/images/nomad-data-model.png new file mode 100644 index 000000000..8b8d70739 Binary files /dev/null and b/website/source/assets/images/nomad-data-model.png differ diff --git a/website/source/assets/images/nomad-evaluation-flow.png b/website/source/assets/images/nomad-evaluation-flow.png new file mode 100644 index 000000000..b90ffb3d4 Binary files /dev/null and b/website/source/assets/images/nomad-evaluation-flow.png differ diff --git a/website/source/assets/images/nomad-nouns.png b/website/source/assets/images/nomad-nouns.png deleted file mode 100644 index 16d2b9854..000000000 Binary files a/website/source/assets/images/nomad-nouns.png and /dev/null differ diff --git a/website/source/assets/images/region-arch.png b/website/source/assets/images/region-arch.png deleted file mode 100644 index def83e4bf..000000000 Binary files a/website/source/assets/images/region-arch.png and /dev/null differ diff --git a/website/source/community.html.erb b/website/source/community.html.erb index 95d5ff282..9ac2741b7 100644 --- a/website/source/community.html.erb +++ b/website/source/community.html.erb @@ -34,54 +34,22 @@ the faces below, Nomad is a project by employees actively contribute to Nomad.

-
- -
-

Mitchell Hashimoto (@mitchellh)

-

- Mitchell Hashimoto is the creator of Nomad and works on all - layers of Nomad from the core to backends. In addition to Nomad, - Mitchell is the creator of - Vagrant, - Packer, - Consul, and - Terraform. -

-
-
- +

Armon Dadgar (@armon)

- Armon Dadgar is a creator of Nomad. He works on all aspects of Nomad, - focusing on core architecture and security. Armon is also the creator of - Consul, - Serf, - Terraform, - Statsite, and - Bloomd. -

+ Armon Dadgar is a creator of Nomad. He works on all aspects of Nomad, + focusing on core architecture and security. Armon is also an author or + core contributor to: + Consul, + Serf, + Terraform, + Statsite, + and Bloomd. +

-
- -
- -
-

Jack Pearkes (@pearkes)

-

- Jack Pearkes is the creator of the online interactive demo of Nomad. - He maintains this demo as well as the design and interaction of the - Nomad website. Jack is an employee of HashiCorp and a primary engineer - behind Atlas. - He is also a core committer to - Packer, - Consul, and - Terraform. -

-
-
- -
+
+
diff --git a/website/source/docs/commands/index.html.md.erb b/website/source/docs/commands/index.html.md.erb index 0cee21ce4..91bf0ca5c 100644 --- a/website/source/docs/commands/index.html.md.erb +++ b/website/source/docs/commands/index.html.md.erb @@ -16,7 +16,7 @@ subcommands is in the navigation to the left. The Nomad CLI is a well-behaved command line application. In erroneous cases, a non-zero exit status will be returned. It also responds to `-h` and `--help` -as you'd most likely expect. +as you would most likely expect. To view a list of the available commands at any time, just run Nomad with no arguments. To get help for any specific subcommand, run the subcommand diff --git a/website/source/docs/concepts/auth.html.md b/website/source/docs/concepts/auth.html.md deleted file mode 100644 index 5737cd586..000000000 --- a/website/source/docs/concepts/auth.html.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -layout: "docs" -page_title: "Authentication" -sidebar_current: "docs-concepts-auth" -description: |- - Before performing any operation with Nomad, the connecting client must be authenticated. ---- - -# Authentication - -Before performing any operation with Nomad, the connecting client must be -_authenticated_. Authentication is the process of verifying a person or -machine is who they say they are and assigning an identity to them. This -identity is then used when making requests with Nomad. - -Authentication in Nomad is pluggable via authentication backends. This -allows you to authenticate with Nomad using a method that works best for your -organization. For example, you can authenticate using GitHub, certs, etc. - -## Authentication Backends - -There are many authentication backends available for Nomad. They -are enabled using `vault auth-enable`. After they're enabled, you can -learn more about them using `vault path-help auth/`. For example, -if you enable GitHub, you can use `vault path-help auth/github` to learn more -about how to configure it and login. - -Multiple authentication backends can be enabled, but only one is required -to gain authentication. It is not currently possible to force a user through -multiple authentication backends to gain access. - -This allows you to enable human-friendly as well as machine-friendly -backends at the same time. For example, for humans you might use the -"github" auth backend, and for machines you might use the "app-id" backend. - -## Tokens - -There is an [entire page dedicated to tokens](/docs/concepts/tokens.html), -but it is important to understand that authentication works by verifying -your identity and then generating a token to associate with that identity. - -For example, even though you may authenticate using something like GitHub, -Nomad generates a unique access token for you to use for future requests. -The CLI automatically attaches this token to requests, but if you're using -the API you'll have to do this manually. - -This token given for authentication with any backend can also be used -with the full set of token commands, such as creating new sub-tokens, -revoking tokens, and renewing tokens. This is all covered on the -[token concepts page](/docs/concepts/tokens.html). - -## Authenticating - -#### Via the CLI - -To authenticate with the CLI, `vault auth` is used. This supports many -of the built-in authentication methods. For example, with GitHub: - -``` -$ vault auth -method=github token= -... -``` - -After authenticating, you will be logged in. The CLI command will also -output your raw token. This token is used for revocation and renewal. -As the user logging in, the primary use case of the token is renewal, -covered below in the "Auth Leases" section. - -To determine what variables are needed for an authentication method, -supply the `-method` flag without any additional arguments and help -will be shown. - -If you're using a method that isn't supported via the CLI, then the API -must be used. - -#### Via the API - -API authentication is generally used for machine authentication. Each -auth backend implements is own login endpoint. Use the `vault path-help` -mechanism to find the proper endpoint. - -For example, the GitHub login endpoint is located at `auth/github/login`. -And to determine the arguments needed, `vault path-help auth/github/login` can -be used. - -## Auth Leases - -Just like secrets, identities have -[leases](/docs/concepts/lease.html) associated with them. This means that -you must reauthenticate after the given lease period to continue accessing -Nomad. - -To set the lease associated with an identity, reference the help for -the specific authentication backend in use. It is specific to each backend -how leasing is implemented. - -And just like secrets, identities can be renewed without having to -completely reauthenticate. Just use `vault token-renew ` with the -token associated with your identity to renew it. diff --git a/website/source/docs/concepts/dev-server.html.md b/website/source/docs/concepts/dev-server.html.md deleted file mode 100644 index f579e4a3b..000000000 --- a/website/source/docs/concepts/dev-server.html.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -layout: "docs" -page_title: "Dev Server Mode" -sidebar_current: "docs-concepts-devserver" -description: |- - The dev server in Nomad can be used for development or to experiment with Nomad. ---- - -# "Dev" Server Mode - -You can start Nomad as a server in "dev" mode like so: `vault server -dev`. -This dev-mode server requires no further setup, and your local `vault` CLI will -be authenticated to talk to it. This makes it easy to experiment with Nomad or -start a Nomad instance for development. Every feature of Nomad is available in -"dev" mode. The `-dev` flag just short-circuits a lot of setup to insecure -defaults. - -~> **Warning:** Never, ever, ever run a "dev" mode server in production. -It is insecure and will lose data on every restart (since it stores data -in-memory). It is only made for development or experimentation. - -## Properties - -The properties of the dev server: - - * **Initialized and unsealed** - The server will be automatically initialized - and unsealed. You don't need to use `vault unseal`. It is ready for use - immediately. - - * **In-memory storage** - All data is stored (encrypted) in-memory. Nomad - server doesn't require any file permissions. - - * **Bound to local address without TLS** - The server is listening on - `127.0.0.1:8200` (the default server address) _without_ TLS. - - * **Automatically Authenticated** - The server stores your root access - token so `vault` CLI access is ready to go. If you are accessing Nomad - via the API, you'll need to authenticate using the token printed out. - - * **Single unseal key** - The server is initialized with a single unseal - key. The Nomad is already unsealed, but if you want to experiment with - seal/unseal, then only the single outputted key is required. - -## Use Case - -The dev server should be used for experimentation with Nomad features, such -as different authentication backends, secret backends, audit backends, etc. -If you're new to Nomad, you may want to pick up with [Your First -Secret](http://vaultproject.io/intro/getting-started/first-secret.html) in -our getting started guide. - -In addition to experimentation, the dev server is very easy to automate -for development environments. diff --git a/website/source/docs/concepts/ha.html.markdown b/website/source/docs/concepts/ha.html.markdown index 630d7f45c..355a4b22e 100644 --- a/website/source/docs/concepts/ha.html.markdown +++ b/website/source/docs/concepts/ha.html.markdown @@ -12,31 +12,4 @@ Nomad supports multi-server mode for high availability. This mode protects against outages by running multiple Nomad servers. High availability mode is automatically enabled when using a storage backend that supports it. -You can tell if a backend supports high availability mode ("HA") by -starting the server and seeing if "(HA available)" is outputted next to -the backend information. If it is, then HA will begin happening automatically. - -To be highly available, Nomad elects a leader and does request forwarding to -the leader. Due to this architecture, HA does not enable increased scalability. -In general, the bottleneck of Nomad is the storage backend itself, not -Nomad core. For example: to increase scalability of Nomad with Consul, you -would scale Consul instead of Nomad. - -In addition to using a backend that supports HA, you have to configure -Nomad with an _advertise address_. This is the address that Nomad advertises -to other Nomad servers in the cluster for request forwarding. By default, -Nomad will use the first private IP address it finds, but you can override -this to any address you want. - -## Backend Support - -Currently there are several backends that support high availability mode, -including Consul, ZooKeeper and etcd. These may change over time, and the -[configuration page](/docs/config/index.html) should be referenced. - -The Consul backend is the recommended HA backend, as it is used in production -by HashiCorp and it's customers with commercial support. - -If you're interested in implementing another backend or adding HA support -to another backend, we'd love your contributions. Adding HA support -requires implementing the `physical.HABackend` interface for the storage backend. +TODO: Document HA diff --git a/website/source/docs/concepts/lease.html.md b/website/source/docs/concepts/lease.html.md deleted file mode 100644 index fc89b9e79..000000000 --- a/website/source/docs/concepts/lease.html.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -layout: "docs" -page_title: "Lease, Renew, and Revoke" -sidebar_current: "docs-concepts-lease" -description: |- - Nomad provides a lease with every secret. When this lease is expired, Nomad will revoke that secret. ---- - -# Lease, Renew, and Revoke - -With every secret and authentication token, Nomad provides a _lease_: -an amount of time that Nomad promises that the data will be valid for. -Once the lease is up, Nomad can automatically revoke the data, and the -consumer of the secret can no longer be certain that it is valid. - -The benefit should be clear: consumers of secrets need to check in with -Nomad routinely to either renew the lease (if allowed) or request a -replacement secret. This makes the Nomad audit logs more valuable and -also makes key rolling a lot easier. - -All secrets in Nomad are required to have a lease. Even if the data is -meant to be valid for eternity, a lease is required to force the consumer -to check in routinely. - -In addition to renewals, a lease can be _revoked_. When a lease is revoked, -it invalidates that secret immediately and prevents any further renewals. -For -[dynamic secrets](#), -the secrets themselves are often immediately disabled. For example, with -the -[AWS secret backend](/docs/secrets/aws/index.html), the access keys will -be deleted from AWS the moment a secret is revoked. This renders the access -keys invalid from that point forward. - -Revocation can happen manually via the API or `vault revoke`, or automatically -by Nomad. When a lease is expired, Nomad will automatically revoke that -lease. - -## Lease IDs - -When reading a secret, such as via `vault read`, Nomad always returns -a `lease_id`. This is the ID used with commands such as `vault renew` and -`vault revoke` to manage the lease of the secret. - -## Lease Durations and Renewal - -Along with the lease ID, a _lease duration_ can be read. The lease duration -is the time in seconds that the lease is valid for. A consumer of this -secret must renew the lease within that time. - -When renewing the lease, the user can request a specific amount of time -from now to extend the lease. For example: `vault renew my-lease-id 3600` -would request to extend the lease of "my-lease-id" by 1 hour (3600 seconds). - -The requested increment is completely advisory. The backend in charge -of the secret can choose to completely ignore it. For most secrets, the -backend does its best to respect the increment, but often limits it to -ensure renewals every so often. - -As a result, the return value of renews should be carefully inspected -to determine what the new lease is. - -## Prefix-based Revocation - -In addition to revoking a single secret, operators with proper access -control can revoke multiple secrets based on their lease ID prefix. - -Lease IDs are structured in a way that their prefix is always the path -where the secret was requested from. This lets you revoke trees of -secrets. For example, to revoke all AWS access keys, you can do -`vault revoke -prefix aws/`. - -This is very useful if there is an intrusion within a specific system: -all secrets of a specific backend or a certain configured backend can -be revoked quickly and easily. diff --git a/website/source/docs/concepts/policies.html.md b/website/source/docs/concepts/policies.html.md deleted file mode 100644 index 5e9d38184..000000000 --- a/website/source/docs/concepts/policies.html.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -layout: "docs" -page_title: "Policies" -sidebar_current: "docs-concepts-policies" -description: |- - Policies are how authorization is done in Nomad, allowing you to restrict which parts of Nomad a user can access. ---- - -# Access Control Policies - -After [authenticating](/docs/concepts/auth.html) with Nomad, the -next step is authorization. This is the process of determining what -a user is allowed to do. Authorization is unified in Nomad in the form -of _policies_. - -Policies are [HCL](https://github.com/hashicorp/hcl) or JSON documents -that describe what parts of Nomad a user is allowed to access. An example -of a policy is shown below: - -```javascript -path "sys/*" { - policy = "deny" -} - -path "secret/*" { - policy = "write" -} - -path "secret/foo" { - policy = "read" -} - -path "secret/super-secret" { - policy = "deny" -} -``` - -Policies use path based matching to apply rules. A policy may be an exact -match, or might be a glob pattern which uses a prefix. The default policy -is always deny so if a path isn't explicitly allowed, Nomad will reject access to it. -This works well due to Nomad's architecture of being like a filesystem: -everything has a path associated with it, including the core configuration -mechanism under "sys". - -~> Policy paths are matched using the most specific defined policy. This may -be an exact match or the longest-prefix match of a glob. This means if you -define a policy for `"secret/foo*"`, the policy would also match `"secret/foobar"`. -The glob character is only supported at the end of the path specification. - -## Policies - -Allowed policies for a path are: - - * `deny` - No access allowed. Highest precedence. - - * `sudo` - Read, write, and root access to a path. - - * `write` - Read, write access to a path. - - * `read` - Read-only access to a path. - -The only non-obvious policy is "sudo". Some routes within Nomad and mounted -backends are marked as _root_ paths. Clients aren't allowed to access root -paths unless they are a root user (have the special policy "root") or -have access to that path with the "sudo" policy. - -For example, modifying the audit log backends is done via root paths. -Only root or "sudo" privilege users are allowed to do this. - -## Root Policy - -The "root" policy is a special policy that can not be modified or removed. -Any user associated with the "root" policy becomes a root user. A root -user can do _anything_ within Nomad. - -There always exists at least one root user (associated with the token -when initializing a new server). After this root user, it is recommended -to create more strictly controlled users. The original root token should -be protected accordingly. - -## Managing Policies - -Policy management can be done via the API or CLI. The CLI commands are -`vault policies` and `vault policy-write`. Please see the help associated -with these commands for more information. They are very easy to use. - -## Associating Policies - -To associate a policy with a user, you must consult the documentation for -the authentication backend you're using. - -For tokens, they are associated at creation time with `vault token-create` -and the `-policy` flags. Child tokens can be associated with a subset of -a parent's policies. Root users can assign any policies. - -There is no way to modify the policies associated with an active -identity. The identity must be revoked and reauthenticated to receive -the new policy list. - -If an _existing_ policy is modified, the modifications propagate -to all associated users instantly. The above paragraph is more specifically -stating that you can't add new or remove policies associated with an -active identity. - -## Changes from 0.1 - -In Nomad versions prior to 0.2, the ACL policy language had a slightly -different specification and semantics. The current specification requires -that glob behavior explicitly be specified by adding the `*` character to -the end of a path. Previously, all paths were glob based matches and no -exact match could be specified. - -The other change is that deny had the lowest precedence. This meant if there -were two policies being merged (e.g. "ops" and "prod") and they had a conflicting -policy like: - -``` -path "sys/seal" { - policy = "deny" -} - -path "sys/seal" { - policy = "read" -} -``` - -The merge would previously give the "read" higher precedence. The current -version of Nomad prioritizes the explicit deny, so that the "deny" would -take precedence. - -To make all Nomad 0.1 policies compatible with Nomad 0.2, the explicit -glob character must be added to all the path prefixes. - diff --git a/website/source/docs/concepts/seal.html.md b/website/source/docs/concepts/seal.html.md deleted file mode 100644 index 165536bfd..000000000 --- a/website/source/docs/concepts/seal.html.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -layout: "docs" -page_title: "Seal/Unseal" -sidebar_current: "docs-concepts-seal" -description: |- - A Nomad must be unsealed before it can access its data. Likewise, it can be sealed to lock it down. ---- - -# Seal/Unseal - -When a Nomad server is started, it starts in a _sealed_ state. In this -state, Nomad is configured to know where and how to access the physical -storage, but doesn't know how to decrypt any of it. - -_Unsealing_ is the process of constructing the master key necessary to -read the decryption key to decrypt the data, allowing access to the Nomad. - -Prior to unsealing, almost no operations are possible with Nomad. For -example authentication, managing the mount tables, etc. are all not possible. -The only possible operations are to unseal the Nomad and check the status -of the unseal. - -## Why? - -The data stored by Nomad is stored encrypted. Nomad needs the -_encryption key_ in order to decrypt the data. The encryption key is -also stored with the data, but encrypted with another encryption key -known as the _master key_. The master key isn't stored anywhere. - -Therefore, to decrypt the data, Nomad must decrypt the encryption key -which requires the master key. Unsealing is the process of reconstructing -this master key. - -Instead of distributing this master key as a single key to an operator, -Nomad uses an algorithm known as -[Shamir's Secret Sharing](http://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing) -to split the key into shards. A certain threshold of shards is required to -reconstruct the master key. - -This is the _unseal_ process: the shards are added one at a time (in any -order) until enough shards are present to reconstruct the key and -decrypt the data. - -## Unsealing - -The unseal process is done by running `vault unseal` or via the API. -This process is stateful: each key can be entered via multiple mechanisms -on multiple computers and it will work. This allows each shard of the master -key to be on a distinct machine for better security. - -Once a Nomad is unsealed, it remains unsealed until one of two things happens: - - 1. It is resealed via the API (see below). - - 2. The server is restarted. - --> **Note:** Unsealing makes the process of automating a Nomad install -difficult. Automated tools can easily install, configure, and start Nomad, -but unsealing it is a very manual process. We have plans in the future to -make it easier. For the time being, the best method is to manually unseal -multiple Nomad servers in [HA mode](/docs/concepts/ha.html). Use a tool such -as Consul to make sure you only query Nomad servers that are unsealed. - -## Sealing - -There is also an API to seal the Nomad. This will throw away the encryption -key and require another unseal process to restore it. Sealing only requires -a single operator with root privileges. - -This way, if there is a detected intrusion, the Nomad data can be locked -quickly to try to minimize damages. It can't be accessed again without -access to the master key shards. diff --git a/website/source/docs/concepts/tokens.html.md b/website/source/docs/concepts/tokens.html.md deleted file mode 100644 index f52f3f473..000000000 --- a/website/source/docs/concepts/tokens.html.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -layout: "docs" -page_title: "Tokens" -sidebar_current: "docs-concepts-tokens" -description: |- - Tokens are a core authentication method in Nomad. Child tokens, token-based revocation, and more. ---- - -# Tokens - -Tokens are the core method for _authentication_ within Nomad. Tokens -can be used directly or [authentication backends](/docs/concepts/auth.html) -can be used to dynamically generate tokens based on external identities. - -If you've gone through the getting started guide, you probably noticed that -`vault server -dev` (or `vault init` for a non-dev server) outputs an -initial "root token." This is the first method of authentication for Nomad. -It is also the only authentication backend that cannot be disabled. - -As stated in the [authentication concepts](/docs/concepts/auth.html), -all external authentication mechanisms such as GitHub map down to dynamically -created tokens. These tokens have all the same properties as a normal -manually created token. - -On this page, we'll show you how to create and manage tokens. - -## Token Creation - -Tokens are created via the API or using `vault token-create` from the CLI. -This will create a new token that is a child of the currently authenticated -token. As a child, the new token will automatically be revoked if the parent -is revoked. - -If you're logged in as root, you can create an _orphan_ token by -specifying the `-orphan` flag. An orphan token has no parent, and therefore -when your token is revoked, it will not revoke the orphan. - -Metadata associated with the token with `-metadata` is used to annotate -the token with information that is added to the audit log. - -Finally, the `-policy` flag can be used to set the policies associated -with the token. Learn more about policies on the -[policies concepts](/docs/concepts/policies.html) page. - -## Token Leases - -Every token has a lease associated with it. These leases behave in much -the same way as [leases for secrets](/docs/concepts/lease.html). After -the lease period is up, the token will no longer function. In addition -to no longer functioning, Nomad will revoke it. - -In order to avoid your token being revoked, the `vault token-renew` -command should be used to renew the lease on the token periodically. - -After a token is revoked, all of the secrets in use by that token will -also be revoked. Therefore, if a user requests AWS access keys, for example, -then after the token expires the AWS access keys will also be expired even -if they had remaining lease time. diff --git a/website/source/docs/config/index.html.md b/website/source/docs/config/index.html.md index 7528a9b20..14d7d18c7 100644 --- a/website/source/docs/config/index.html.md +++ b/website/source/docs/config/index.html.md @@ -6,12 +6,14 @@ description: |- Nomad server configuration reference. --- -# Server Configuration +# Nomad Configuration Outside of development mode, Nomad servers are configured using a file. The format of this file is [HCL](https://github.com/hashicorp/hcl) or JSON. An example configuration is shown below: +TODO: Document Nomad configuration. Examples below stolen from Vault. + ```javascript backend "consul" { address = "127.0.0.1:8500" @@ -66,177 +68,3 @@ the Nomad executable access to the `mlock` syscall on Linux systems: ```shell sudo setcap cap_ipc_lock=+ep $(readlink -f $(which vault)) ``` - -## Backend Reference - -For the `backend` section, the supported backends are shown below. -Nomad requires that the backend itself will be responsible for backups, -durability, etc. - - * `consul` - Store data within [Consul](http://www.consul.io). This - backend supports HA. It is the most recommended backend for Nomad - and has been shown to work at high scale under heavy load. - - * `etcd` - Store data within [etcd](https://coreos.com/etcd/). - This backend supports HA. - - * `zookeeper` - Store data within [Zookeeper](https://zookeeper.apache.org/). - This backend supports HA. - - * `s3` - Store data within an S3 bucket [S3](http://aws.amazon.com/s3/). - This backend does not support HA. - - * `mysql` - Store data within MySQL. This backend does not support HA. - - * `inmem` - Store data in-memory. This is only really useful for - development and experimentation. Data is lost whenever Nomad is - restarted. - - * `file` - Store data on the filesystem using a directory structure. - This backend does not support HA. - -#### Common Backend Options - -All backends support the following options: - - * `advertise_addr` (optional) - For backends that support HA, this - is the address to advertise to other Nomad servers in the cluster - for request forwarding. Most HA backends will attempt to determine - the advertise address if not provided. - -#### Backend Reference: Consul - -For Consul, the following options are supported: - - * `path` (optional) - The path within Consul where data will be stored. - Defaults to "vault/". - - * `address` (optional) - The address of the Consul agent to talk to. - Defaults to the local agent address, if available. - - * `scheme` (optional) - "http" or "https" for talking to Consul. - - * `datacenter` (optional) - The datacenter within Consul to write to. - This defaults to the local datacenter. - - * `token` (optional) - An access token to use to write data to Consul. - - * `tls_skip_verify` (optional) - If non-empty, then TLS host verification - will be disabled for Consul communication. - Defaults to false. - - The following settings should be set according to your [Consul encryption settings](https://www.consul.io/docs/agent/encryption.html): - - * `tls_ca_file` (optional) - The path to the CA certificate used for Consul communication. - Defaults to system bundle if not specified. - Set accordingly to the [ca_file](https://www.consul.io/docs/agent/options.html#ca_file) setting in Consul. - - * `tls_cert_file` (optional) - The path to the certificate for Consul communication. - Set accordingly to the [cert_file](https://www.consul.io/docs/agent/options.html#cert_file) setting in Consul. - - * `tls_key_file` (optional) - The path to the private key for Consul communication. - Set accordingly to the [key_file](https://www.consul.io/docs/agent/options.html#key_file) setting in Consul. - -#### Backend Reference: Zookeeper - -For Zookeeper, the following options are supported: - - * `path` (optional) - The path within Zookeeper where data will be stored. - Defaults to "vault/". - - * `address` (optional) - The address(es) of the Zookeeper instance(s) to talk to. - Can be comma separated list (host:port) of many Zookeeper instances. - Defaults to "localhost:2181" if not specified. - -#### Backend Reference: etcd - -For etcd, the following options are supported: - - * `path` (optional) - The path within etcd where data will be stored. - Defaults to "vault/". - - * `address` (optional) - The address(es) of the etcd instance(s) to talk to. - Can be comma separated list (protocol://host:port) of many etcd instances. - Defaults to "http://localhost:4001" if not specified. - -#### Backend Reference: S3 - -For S3, the following options are supported: - - * `bucket` (required) - The name of the S3 bucket to use. - - * `access_key` - (required) The AWS access key. It must be provided, but it can also be sourced from the AWS_ACCESS_KEY_ID environment variable. - - * `secret_key` - (required) The AWS secret key. It must be provided, but it can also be sourced from the AWS_SECRET_ACCESS_KEY environment variable. - - * `session_token` - (optional) The AWS session_token. It can also be sourced from the AWS_SESSION_TOKEN environment variable. - - * `region` (optional) - The AWS region. It can be sourced from the AWS_DEFAULT_REGION environment variable and will default to "us-east-1" if not specified. - -#### Backend Reference: MySQL - -The MySQL backend has the following options: - - * `username` (required) - The MySQL username to connect with. - - * `password` (required) - The MySQL password to connect with. - - * `address` (optional) - The address of the MySQL host. Defaults to - "127.0.0.1:3306. - - * `database` (optional) - The name of the database to use. Defaults to "vault". - - * `table` (optional) - The name of the table to use. Defaults to "vault". - - * `tls_ca_file` (optional) - The path to the CA certificate to connect using TLS - -#### Backend Reference: Inmem - -The in-memory backend has no configuration options. - -#### Backend Reference: File - -The file backend has the following options: - - * `path` (required) - The path on disk to a directory where the - data will be stored. - -## Listener Reference - -For the `listener` section, the only supported listener currently -is "tcp". Regardless of future plans, this is the recommended listener, -since it allows for HA mode. - -The supported options are: - - * `address` (optional) - The address to bind to for listening. This - defaults to "127.0.0.1:8200". - - * `tls_disable` (optional) - If non-empty, then TLS will be disabled. - This is an opt-in; Nomad assumes by default that TLS will be used. - - * `tls_cert_file` (required unless disabled) - The path to the certificate - for TLS. - - * `tls_key_file` (required unless disabled) - The path to the private key - for the certificate. - - * `tls_min_version` (optional) - **(Nomad > 0.2)** If provided, specifies - the minimum supported version of TLS. Accepted values are "tls10", "tls11" - or "tls12". This defaults to "tls12". WARNING: TLS 1.1 and lower - are generally considered less secure; avoid using these if - possible. - -## Telemetry Reference - -For the `telemetry` section, there is no resource name. All configuration -is within the object itself. - -* `statsite_address` (optional) - An address to a [Statsite](https://github.com/armon/statsite) - instances for metrics. This is highly recommended for production usage. - -* `statsd_address` (optional) - This is the same as `statsite_address` but - for StatsD. - -* `disable_hostname` (optional) - Whether or not to prepend runtime telemetry - with the machines hostname. This is a global option. Defaults to false. diff --git a/website/source/docs/drivers/custom.html.md b/website/source/docs/drivers/custom.html.md index d82a35823..53f30939e 100644 --- a/website/source/docs/drivers/custom.html.md +++ b/website/source/docs/drivers/custom.html.md @@ -8,10 +8,10 @@ description: |- # Custom Drivers -Nomad doesn't currently support the creation of custom secret backends. +Nomad does not currently support the creation of custom secret backends. The primary reason is because we want to ensure the core of Nomad is secure before attempting any sort of plug-in system. We're interested -in supporting custom secret backends, but don't yet have a clear strategy +in supporting custom secret backends, but do not yet have a clear strategy or timeline to do. In the mean time, you can use the diff --git a/website/source/docs/drivers/index.html.md b/website/source/docs/drivers/index.html.md index 082d76920..4175c7e8e 100644 --- a/website/source/docs/drivers/index.html.md +++ b/website/source/docs/drivers/index.html.md @@ -28,11 +28,11 @@ a write, etc. This is a lot like a normal filesystem. The "aws" backend, on the other hand, behaves differently. When you write to `aws/config/root`, it expects a certain format and stores that -information as configuration. You can't read from this path. When you +information as configuration. You cannot read from this path. When you read from `aws/`, it looks up an IAM policy named `` and -generates AWS access credentials on demand and returns them. It doesn't -behave at all like a typical filesystem: you're not simply storing and -retrieving values, you're interacting with an API. +generates AWS access credentials on demand and returns them. It does not +behave at all like a typical filesystem: you are not simply storing and +retrieving values, you are interacting with an API. ## Mounting/Unmounting Secret Backends @@ -53,7 +53,7 @@ with regards to mounting: * **Remount** - This moves the mount point for an existing secret backend. This revokes all secrets, since secret leases are tied to the path they - were created at. The data stored for the backend won't be deleted. + were created at. The data stored for the backend will not be deleted. Once a secret backend is mounted, you can interact with it directly at its mount point according to its own API. You can use the `vault path-help` @@ -68,8 +68,8 @@ like a [chroot](http://en.wikipedia.org/wiki/Chroot). Whenever a secret backend is mounted, a random UUID is generated. This becomes the data root for that backend. Whenever that backend writes to the physical storage layer, it is prefixed with that UUID folder. Since -the Nomad storage layer doesn't support relative access (such as `..`), +the Nomad storage layer does not support relative access (such as `..`), this makes it impossible for a mounted backend to access any other data. This is an important security feature in Nomad: even a malicious backend -can't access the data from any other backend. +cannot access the data from any other backend. diff --git a/website/source/docs/http/index.html.md b/website/source/docs/http/index.html.md index 6eaa6d27f..16145bce1 100644 --- a/website/source/docs/http/index.html.md +++ b/website/source/docs/http/index.html.md @@ -12,137 +12,4 @@ The Nomad HTTP API gives you full access to Nomad via HTTP. Every aspect of Nomad can be controlled via this API. The Nomad CLI uses the HTTP API to access Nomad. -## Version Prefix - -All API routes are prefixed with `/v1/`. - -This documentation is only for the v1 API. - -~> **Backwards compatibility:** At the current version, Nomad does -not yet promise backwards compatibility even with the v1 prefix. We'll -remove this warning when this policy changes. We expect we'll reach API -stability by Nomad 0.3. - -## Transport - -The API is expected to be accessed over a TLS connection at -all times, with a valid certificate that is verified by a well -behaved client. It is possible to disable TLS verification for -listeners, however, so API clients should expect to have to do both -depending on user settings. - -## Authentication - -Once the Nomad is unsealed, every other operation requires -a _client token_. A user may have a client token explicitly. -The client token must be sent as the `token` cookie or the -`X-Nomad-Token` HTTP header. - -Otherwise, a client token can be retrieved via -[authentication backends](/docs/auth/index.html). - -Each authentication backend will have one or more unauthenticated -login endpoints. These endpoints can be reached without any authentication, -and are used for authentication itself. These endpoints are specific -to each authentication backend. - -Login endpoints for authentication backends that generate an identity -will be sent down with a `Set-Cookie` header as well as via JSON. If you have a -well-behaved HTTP client, then authentication information will -automatically be saved and sent to the Nomad API. - -## Reading and Writing Secrets - -Reading a secret via the HTTP API is done by issuing a GET using the -following URL: - -```text -/v1/secret/foo -``` - -This maps to `secret/foo` where `foo` is the key in the `secret/` backend/ - -Here is an example of reading a secret using cURL: - -```shell -curl \ - -H "X-Nomad-Token: f3b09679-3001-009d-2b80-9c306ab81aa6" \ - -X GET \ - http://127.0.0.1:8200/v1/secret/foo -``` - -To write a secret, issue a POST on the following URL: - -```text -/v1/secret/foo -``` - -with a JSON body like: - -```javascript -{ - "value": "bar" -} -``` - -Here is an example of writing a secret using cURL: - -```shell -curl \ - -H "X-Nomad-Token: f3b09679-3001-009d-2b80-9c306ab81aa6" \ - -H "Content-Type: application/json" \ - -X POST \ - -d '{"value":"bar"}' \ - http://127.0.0.1:8200/v1/secret/baz -``` - -For more examples, please look at the Nomad API client. - -## Help - -To retrieve the help for any API within Nomad, including mounted -backends, credential providers, etc. then append `?help=1` to any -URL. If you have valid permission to access the path, then the help text -will be returned with the following structure: - -```javascript -{ - "help": "help text" -} -``` - -## Error Response - -A common JSON structure is always returned to return errors: - -```javascript -{ - "errors": [ - "message", - "another message" - ] -} -``` - -This structure will be sent down for any HTTP status greater than -or equal to 400. - -## HTTP Status Codes - -The following HTTP status codes are used throughout the API. - -- `200` - Success with data. -- `204` - Success, no data returned. -- `400` - Invalid request, missing or invalid data. See the - "validation" section for more details on the error response. -- `401` - Unauthorized, your authentication details are either - incorrect or you don't have access to this feature. -- `404` - Invalid path. This can both mean that the path truly - doesn't exist or that you don't have permission to view a - specific path. We use 404 in some cases to avoid state leakage. -- `429` - Rate limit exceeded. Try again after waiting some period - of time. -- `500` - Internal server error. An internal error has occurred, - try again later. If the error persists, report a bug. -- `503` - Nomad is down for maintenance or is currently sealed. - Try again later. +TODO: Document Nomand API diff --git a/website/source/docs/http/libraries.html.md b/website/source/docs/http/libraries.html.md deleted file mode 100644 index 46ad58e65..000000000 --- a/website/source/docs/http/libraries.html.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: Libraries" -sidebar_current: "docs-http-libraries" -description: |- - List of official and community contributed libraries for interacting with the Nomad HTTP API. ---- - -# Libraries - -The programming libraries listed on this page can be used to consume the API more conveniently. -Some are officially maintained while others are provided by the community. - -## Go - -* [Nomad Go Client](https://github.com/hashicorp/vault/tree/master/api) (official) - -## Ruby - -* [Nomad Ruby Client](https://github.com/hashicorp/vault-ruby) (official) - -## Python - -* [HVAC](https://github.com/ianunruh/hvac) - -## Node.js - -* [node-vault](https://github.com/kr1sp1n/node-vault) -* [vaulted](https://github.com/chiefy/vaulted) diff --git a/website/source/docs/http/sys-audit.html.md b/website/source/docs/http/sys-audit.html.md deleted file mode 100644 index 7040e2004..000000000 --- a/website/source/docs/http/sys-audit.html.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/audit" -sidebar_current: "docs-http-audits-audits" -description: |- - The `/sys/audit` endpoint is used to enable and disable audit backends. ---- - -# /sys/audit - -## GET - -
-
Description
-
- Lists all the available policies. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- - ```javascript - { - "file": { - "type: "file", - "description: "Store logs in a file", - "options": { - "path": "/var/log/file" - } - } - } - ``` - -
-
- -## PUT - -
-
Description
-
- Enable an audit backend. -
- -
Method
-
PUT
- -
URL
-
`/sys/audit/`
- -
Parameters
-
-
    -
  • - type - required - The type of the audit backend. -
    • -
  • - description - optional - A description of the audit backend for operators. -
    • -
  • - options - optional - An object of options to configure the backend. This is - dependent on the backend type. Please consult the documentation - for the backend type you intend to use. -
    • -
-
- -
Returns
-
`204` response code. -
-
- -## DELETE - -
-
Description
-
- Disable the given audit backend. -
- -
Method
-
DELETE
- -
URL
-
`/sys/audit/`
- -
Parameters
-
None -
- -
Returns
-
`204` response code. -
-
diff --git a/website/source/docs/http/sys-auth.html.md b/website/source/docs/http/sys-auth.html.md deleted file mode 100644 index 99e22eca5..000000000 --- a/website/source/docs/http/sys-auth.html.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/auth" -sidebar_current: "docs-http-auth-auth" -description: |- - The `/sys/auth` endpoint is used to manage auth backends in Nomad. ---- - -# /sys/auth - -## GET - -
-
Description
-
- Lists all the enabled auth backends. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- - ```javascript - { - "github": { - "type": "github", - "description": "GitHub auth" - } - } - ``` - -
-
- -## POST - -
-
Description
-
- Enable a new auth backend. The auth backend can be accessed - and configured via the mount point specified in the URL. This - mount point will be exposed under the `auth` prefix. For example, - enabling with the `/sys/auth/foo` URL will make the backend - available at `/auth/foo`. -
- -
Method
-
POST
- -
URL
-
`/sys/auth/`
- -
Parameters
-
-
    -
  • - type - required - The name of the auth backend type, such as "github" -
    • -
  • - description - optional - A human-friendly description of the auth backend. -
    • -
-
- -
Returns
-
`204` response code. -
-
- -## DELETE - -
-
Description
-
- Disable the auth backend at the given mount point. -
- -
Method
-
DELETE
- -
URL
-
`/sys/auth/`
- -
Parameters
-
None -
- -
Returns
-
`204` response code. -
-
diff --git a/website/source/docs/http/sys-health.html.md b/website/source/docs/http/sys-health.html.md deleted file mode 100644 index 05d24164b..000000000 --- a/website/source/docs/http/sys-health.html.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/health" -sidebar_current: "docs-http-debug-health" -description: |- - The '/sys/health' endpoint is used to check the health status of Nomad. ---- - -# /sys/health - -
-
Description
-
- Returns the health status of Nomad. This matches the semantics of a Consul HTTP health - check and provides a simple way to monitor the health of a Nomad instance. -
- -
Method
-
GET
- -
Parameters
-
-
    -
  • - standbyok - optional - A query parameter provided to indicate that being a standby should - still return a 200 status code instead of the standard 429 status code. -
    • -
-
- -
Returns
-
- -``` -{ - "initialized": true, - "sealed": false, - "standby": false -} -``` - - Status Codes: - - * `200` if initialized, unsealed and active. - * `429` if unsealed and standby. - * `500` if not initialized or sealed. -
-
diff --git a/website/source/docs/http/sys-init.html.md b/website/source/docs/http/sys-init.html.md deleted file mode 100644 index f1d79be73..000000000 --- a/website/source/docs/http/sys-init.html.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/init" -sidebar_current: "docs-http-sys-init" -description: |- - The '/sys/init' endpoint is used to initialize a new Nomad. ---- - -# /sys/init - -## GET - -
-
Description
-
- Return the initialization status of a Nomad. -
- -
Method
-
GET
- -
Parameters
-
None
- -
Returns
-
- - ```javascript - { - "initialize": true - } - ``` - -
-
- -## PUT - -
-
Description
-
- Initializes a new Nomad. The Nomad must've not been previously - initialized. -
- -
Method
-
PUT
- -
Parameters
-
-
    -
  • - secret_shares - required - The number of shares to split the master key into. -
    • -
  • - secret_threshold - required - The number of shares required to reconstruct the master key. - This must be less than or equal to secret_shares. -
    • -
-
- -
Returns
-
- A JSON-encoded object including the master keys and initial root token: - - ```javascript - { - "keys": ["one", "two", "three"], - "root_token": "foo" - } - ``` - -
-
diff --git a/website/source/docs/http/sys-key-status.html.md b/website/source/docs/http/sys-key-status.html.md deleted file mode 100644 index 07cc58076..000000000 --- a/website/source/docs/http/sys-key-status.html.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/key-status" -sidebar_current: "docs-http-rotate-key-status" -description: |- - The '/sys/key-status' endpoint is used to query info about the current encryption key of Nomad. ---- - -# /sys/key-status - -
-
Description
-
- Returns information about the current encryption key used by Nomad. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- The "term" parameter is the sequential key number, and "install_time" is the time that - encryption key was installed. - - ```javascript - { - "term": 3, - "install_time": "2015-05-29T14:50:46.223692553-07:00" - } - ``` - -
-
diff --git a/website/source/docs/http/sys-leader.html.md b/website/source/docs/http/sys-leader.html.md deleted file mode 100644 index c1c12970f..000000000 --- a/website/source/docs/http/sys-leader.html.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/leader" -sidebar_current: "docs-http-ha-leader" -description: |- - The '/sys/leader' endpoint is used to check the high availability status and current leader of Nomad. ---- - -# /sys/leader - -
-
Description
-
- Returns the high availability status and current leader instance of Nomad. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- - ```javascript - { - "ha_enabled": true, - "is_self": false, - "leader_address": "https://127.0.0.1:8200/" - } - ``` - -
-
diff --git a/website/source/docs/http/sys-mounts.html.md b/website/source/docs/http/sys-mounts.html.md deleted file mode 100644 index e49076c85..000000000 --- a/website/source/docs/http/sys-mounts.html.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/mounts" -sidebar_current: "docs-http-mounts-mounts" -description: |- - The '/sys/mounts' endpoint is used manage secret backends in Nomad. ---- - -# /sys/mounts - -## GET - -
-
Description
-
- Lists all the mounted secret backends. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- - ```javascript - { - "aws": { - "type": "aws", - "description": "AWS keys" - }, - - "sys": { - "type": "system", - "description": "system endpoint" - } - } - ``` - -
-
- -## POST - -
-
Description
-
- Mount a new secret backend to the mount point in the URL. -
- -
Method
-
POST
- -
URL
-
`/sys/mounts/`
- -
Parameters
-
-
    -
  • - type - required - The name of the backend type, such as "aws" -
    • -
  • - description - optional - A human-friendly description of the mount. -
    • -
-
- -
Returns
-
`204` response code. -
-
- -## DELETE - -
-
Description
-
- Unmount the mount point specified in the URL. -
- -
Method
-
DELETE
- -
URL
-
`/sys/mounts/`
- -
Parameters
-
None -
- -
Returns
-
`204` response code. -
-
diff --git a/website/source/docs/http/sys-policy.html.md b/website/source/docs/http/sys-policy.html.md deleted file mode 100644 index c252fc3a1..000000000 --- a/website/source/docs/http/sys-policy.html.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/policy" -sidebar_current: "docs-http-auth-policy" -description: |- - The `/sys/policy` endpoint is used to manage ACL policies in Nomad. ---- - -# /sys/policy - -## GET - -
-
Description
-
- Lists all the available policies. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- - ```javascript - { - "policies": ["root", "deploy"] - } - ``` - -
-
- -## PUT - -
-
Description
-
- Add or update a policy. Once a policy is updated, it takes effect - immediately to all associated users. -
- -
Method
-
PUT
- -
URL
-
`/sys/policy/`
- -
Parameters
-
-
    -
  • - rules - required - The policy document. -
    • -
-
- -
Returns
-
`204` response code. -
-
- -## DELETE - -
-
Description
-
- Delete the policy with the given name. This will immediately - affect all associated users. -
- -
Method
-
DELETE
- -
URL
-
`/sys/policy/`
- -
Parameters
-
None -
- -
Returns
-
`204` response code. -
-
diff --git a/website/source/docs/http/sys-raw.html.md b/website/source/docs/http/sys-raw.html.md deleted file mode 100644 index 03e5f0030..000000000 --- a/website/source/docs/http/sys-raw.html.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/raw" -sidebar_current: "docs-http-debug-raw" -description: |- - The `/sys/raw` endpoint is access the raw underlying store in Nomad. ---- - -# /sys/raw - -## GET - -
-
Description
-
- Reads the value of the key at the given path. This is the raw path in the - storage backend and not the logical path that is exposed via the mount system. -
- -
Method
-
GET
- -
URL
-
`/sys/raw/`
- -
Parameters
-
- None -
- -
Returns
-
- - ```javascript - { - "value": "{'foo':'bar'}" - } - ``` - -
-
- -## PUT - -
-
Description
-
- Update the value of the key at the given path. This is the raw path in the - storage backend and not the logical path that is exposed via the mount system. -
- -
Method
-
PUT
- -
URL
-
`/sys/raw/`
- -
Parameters
-
-
    -
  • - value - required - The value of the key. -
    • -
-
- -
Returns
-
`204` response code. -
-
- -## DELETE - -
-
Description
-
- Delete the key with given path. This is the raw path in the - storage backend and not the logical path that is exposed via the mount system. -
- -
Method
-
DELETE
- -
URL
-
`/sys/raw/`
- -
Parameters
-
None -
- -
Returns
-
`204` response code. -
-
diff --git a/website/source/docs/http/sys-rekey.html.md b/website/source/docs/http/sys-rekey.html.md deleted file mode 100644 index cfbf2d2d5..000000000 --- a/website/source/docs/http/sys-rekey.html.md +++ /dev/null @@ -1,155 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/rekey/" -sidebar_current: "docs-http-rotate-rekey" -description: |- - The `/sys/rekey/` endpoints are used to rekey the unseal keys for Nomad. ---- - -# /sys/rekey/init - -## GET - -
-
Description
-
- Reads the configuration and progress of the current rekey attempt. -
- -
Method
-
GET
- -
URL
-
`/sys/rekey/init`
- -
Parameters
-
- None -
- -
Returns
-
- If a rekey is started, then "n" is the new shares to generate and "t" is - the threshold required for the new shares. The "progress" is how many unseal - keys have been provided for this rekey, where "required" must be reached to - complete. - - ```javascript - { - "started": true, - "t": 3, - "n": 5, - "progress": 1, - "required": 3 - } - ``` - -
-
- -## PUT - -
-
Description
-
- Initializes a new rekey attempt. Only a single rekey attempt can take place - at a time, and changing the parameters of a rekey requires canceling and starting - a new rekey. -
- -
Method
-
PUT
- -
URL
-
`/sys/rekey/init`
- -
Parameters
-
-
    -
  • - secret_shares - required - The number of shares to split the master key into. -
    • -
  • - secret_threshold - required - The number of shares required to reconstruct the master key. - This must be less than or equal to secret_shares. -
    • -
-
- -
Returns
-
`204` response code. -
-
- -## DELETE - -
-
Description
-
- Cancels any in-progress rekey. This clears the rekey settings as well as any - progress made. This must be called to change the parameters of the rekey. -
- -
Method
-
DELETE
- -
URL
-
`/sys/rekey/init`
- -
Parameters
-
None -
- -
Returns
-
`204` response code. -
-
- -# /sys/rekey/update - -## PUT - -
-
Description
-
- Enter a single master key share to progress the rekey of the Nomad. - If the threshold number of master key shares is reached, Nomad - will complete the rekey. Otherwise, this API must be called multiple - times until that threshold is met. -
- -
Method
-
PUT
- -
URL
-
`/sys/rekey/update`
- -
Parameters
-
-
    -
  • - key - required - A single master share key. -
    • -
-
- -
Returns
-
- A JSON-encoded object indicating completion and if so with the new master keys: - - ```javascript - { - "complete": true, - "keys": ["one", "two", "three"] - } - ``` - -
-
- diff --git a/website/source/docs/http/sys-remount.html.md b/website/source/docs/http/sys-remount.html.md deleted file mode 100644 index 231e77092..000000000 --- a/website/source/docs/http/sys-remount.html.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/remount" -sidebar_current: "docs-http-mounts-remount" -description: |- - The '/sys/remount' endpoint is used remount a mounted backend to a new endpoint. ---- - -# /sys/remount - -
-
Description
-
- Remount an already-mounted backend to a new mount point. -
- -
Method
-
POST
- -
Parameters
-
-
    -
  • - from - required - The previous mount point. -
    • -
  • - to - required - The new mount point. -
    • -
-
- -
Returns
-
`204` response code. -
-
diff --git a/website/source/docs/http/sys-renew.html.md b/website/source/docs/http/sys-renew.html.md deleted file mode 100644 index 2d806e00f..000000000 --- a/website/source/docs/http/sys-renew.html.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/renew" -sidebar_current: "docs-http-lease-renew" -description: |- - The `/sys/renew` endpoint is used to renew secrets. ---- - -# /sys/renew - -
-
Description
-
- Renew a secret, requesting to extend the lease. -
- -
Method
-
PUT
- -
URL
-
`/sys/renew/`
- -
Parameters
-
-
    -
  • - increment - optional - A requested amount of time in seconds to extend the lease. - This is advisory. -
    • -
-
- -
Returns
-
A secret structure. -
-
diff --git a/website/source/docs/http/sys-revoke-prefix.html.md b/website/source/docs/http/sys-revoke-prefix.html.md deleted file mode 100644 index d400de552..000000000 --- a/website/source/docs/http/sys-revoke-prefix.html.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/revoke-prefix" -sidebar_current: "docs-http-lease-revoke-prefix" -description: |- - The `/sys/revoke-prefix` endpoint is used to revoke secrets based on prefix. ---- - -# /sys/revoke-prefix - -
-
Description
-
- Revoke all secrets generated under a given prefix immediately. -
- -
Method
-
PUT
- -
URL
-
`/sys/revoke-prefix/`
- -
Parameters
-
None
- -
Returns
-
A `204` response code. -
-
diff --git a/website/source/docs/http/sys-revoke.html.md b/website/source/docs/http/sys-revoke.html.md deleted file mode 100644 index 04672c75d..000000000 --- a/website/source/docs/http/sys-revoke.html.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/revoke" -sidebar_current: "docs-http-lease-revoke-single" -description: |- - The `/sys/revoke` endpoint is used to revoke secrets. ---- - -# /sys/revoke - -
-
Description
-
- Revoke a secret immediately. -
- -
Method
-
PUT
- -
URL
-
`/sys/revoke/`
- -
Parameters
-
None
- -
Returns
-
A `204` response code. -
-
diff --git a/website/source/docs/http/sys-rotate.html.md b/website/source/docs/http/sys-rotate.html.md deleted file mode 100644 index 7b5b07044..000000000 --- a/website/source/docs/http/sys-rotate.html.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/rotate" -sidebar_current: "docs-http-rotate-rotate" -description: |- - The `/sys/rotate` endpoint is used to rotate the encryption key. ---- - -# /sys/rotate - -## PUT - -
-
Description
-
- Trigger a rotation of the backend encryption key. This is the key that is used - to encrypt data written to the storage backend, and is not provided to operators. - This operation is done online. Future values are encrypted with the new key, while - old values are decrypted with previous encryption keys. -
- -
Method
-
PUT
- -
URL
-
`/sys/rotate`
- -
Parameters
-
- None -
- -
Returns
-
`204` response code. -
-
- diff --git a/website/source/docs/http/sys-seal-status.html.md b/website/source/docs/http/sys-seal-status.html.md deleted file mode 100644 index 5e40ed932..000000000 --- a/website/source/docs/http/sys-seal-status.html.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/seal-status" -sidebar_current: "docs-http-seal-status" -description: |- - The '/sys/seal-status' endpoint is used to check the seal status of a Nomad. ---- - -# /sys/seal-status - -
-
Description
-
- Returns the seal status of the Nomad. -
- -
Method
-
GET
- -
Parameters
-
- None -
- -
Returns
-
- The "t" parameter is the threshold, and "n" is the number of shares. - - ```javascript - { - "sealed": true, - "t": 3, - "n": 5, - "progress": 2 - } - ``` - -
-
diff --git a/website/source/docs/http/sys-seal.html.md b/website/source/docs/http/sys-seal.html.md deleted file mode 100644 index ccedcf099..000000000 --- a/website/source/docs/http/sys-seal.html.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/seal" -sidebar_current: "docs-http-seal-seal" -description: |- - The '/sys/seal' endpoint seals the Nomad. ---- - -# /sys/seal - -
-
Description
-
- Seals the Nomad. -
- -
Method
-
PUT
- -
Parameters
-
- None -
- -
Returns
-
A `204` response code. -
-
diff --git a/website/source/docs/http/sys-unseal.html.md b/website/source/docs/http/sys-unseal.html.md deleted file mode 100644 index abe793ef7..000000000 --- a/website/source/docs/http/sys-unseal.html.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -layout: "http" -page_title: "HTTP API: /sys/seal-unseal" -sidebar_current: "docs-http-seal-unseal" -description: |- - The '/sys/seal-unseal' endpoint is used to unseal the Nomad. ---- - -# /sys/unseal - -
-
Description
-
- Enter a single master key share to progress the unsealing of the Nomad. - If the threshold number of master key shares is reached, Nomad - will attempt to unseal the Nomad. Otherwise, this API must be - called multiple times until that threshold is met. -
- -
Method
-
PUT
- -
Parameters
-
-
    -
  • - key - required - A single master share key. -
    • -
-
-
Returns
-
The same result as `/sys/seal-status`. -
-
diff --git a/website/source/docs/index.html.markdown b/website/source/docs/index.html.markdown index 857f5c05c..149dc28e2 100644 --- a/website/source/docs/index.html.markdown +++ b/website/source/docs/index.html.markdown @@ -3,12 +3,13 @@ layout: "docs" page_title: "Documentation" sidebar_current: "docs-home" description: |- - Welcome to the Nomad documentation! This documentation is more of a reference guide for all available features and options of Nomad. If you're just getting started with Nomad, please start with the introduction and getting started guide instead. + Welcome to the Nomad documentation! This documentation is more of a reference + guide for all available features and options of Nomad. --- # Nomad Documentation -Welcome to the Nomad documentation! This documentation is more of a reference -guide for all available features and options of Nomad. If you're just getting +Welcome to the Nomad documentation! This documentation is a reference guide for +all available features and options of Nomad. If you are just getting started with Nomad, please start with the [introduction and getting started guide](/intro/index.html) instead. diff --git a/website/source/docs/install/index.html.md b/website/source/docs/install/index.html.md index e8d73a20e..b684806b4 100644 --- a/website/source/docs/install/index.html.md +++ b/website/source/docs/install/index.html.md @@ -18,7 +18,7 @@ TLS along with SHA256 sums to verify the binary. To install the precompiled binary, [download](/downloads.html) the appropriate package for your system. -Nomad is currently packaged as a zip file. We don't have any near term +Nomad is currently packaged as a zip file. We do not have any near term plans to provide system packages. Once the zip is downloaded, unzip it into any directory. The @@ -31,13 +31,13 @@ from the command-line, make sure to place it somewhere on your `PATH`. ## Compiling from Source -To compile from source, you'll need [Go](https://golang.org) installed -and configured properly. You'll also need Git. +To compile from source, you will need [Go](https://golang.org) installed +and configured properly. you will also need Git. 1. Clone the Nomad repository into your GOPATH: https://github.com/hashicorp/nomad 1. Verify that the file `$GOPATH/src/github.com/hashicorp/nomad/main.go` - exists. If it doesn't, then you didn't clone Nomad into the proper + exists. If it does not, then you did not clone Nomad into the proper path. 1. Run `make`. This will build Nomad for your current system diff --git a/website/source/docs/internals/architecture.html.md b/website/source/docs/internals/architecture.html.md index 0dd5ef76d..ffefe87c1 100644 --- a/website/source/docs/internals/architecture.html.md +++ b/website/source/docs/internals/architecture.html.md @@ -12,7 +12,7 @@ Nomad is a complex system that has many different pieces. To help both users and build a mental model of how it works, this page documents the system architecture. ~> **Advanced Topic!** This page covers technical details -of Nomad. You don't need to understand these details to +of Nomad. You do not need to understand these details to effectively use Nomad. The details are documented here for those who wish to learn about them without having to go spelunking through the source code. @@ -74,7 +74,7 @@ clarify what is being discussed: Looking at only a single region, at a high level Nomad looks like: -![Regional Architecture](/assets/images/region-arch.png) +[![Regional Architecture](/assets/images/nomad-architecture-region.png)](/assets/images/nomad-architecture-region.png) Within each region, we have both clients and servers. Servers are responsible for accepting jobs from users, managing clients, and [computing task placements](/docs/internals/scheduling.html). @@ -85,7 +85,7 @@ In some cases, for either availability or scalability, you may need to run multi regions. Nomad supports federating multiple regions together into a single cluster. At a high level, this looks like: -![Global Architecture](/assets/images/global-arch.png) +[![Global Architecture](/assets/images/nomad-architecture-global.png)](/assets/images/nomad-architecture-global.png) Regions are fully independent from each other, and do not share jobs, clients or state. They are loosely-coupled using a gossip protocol, which allows users to diff --git a/website/source/docs/internals/consensus.html.md b/website/source/docs/internals/consensus.html.md index aba700507..16b86565c 100644 --- a/website/source/docs/internals/consensus.html.md +++ b/website/source/docs/internals/consensus.html.md @@ -3,54 +3,57 @@ layout: "docs" page_title: "Consensus Protocol" sidebar_current: "docs-internals-consensus" description: |- - Nomad uses a consensus protocol to provide Consistency as defined by CAP. The consensus protocol is based on Raft: In search of an Understandable Consensus Algorithm. For a visual explanation of Raft, see The Secret Lives of Data. + Nomad uses a consensus protocol to provide Consistency as defined by CAP. + The consensus protocol is based on Raft: In search of an Understandable + Consensus Algorithm. For a visual explanation of Raft, see The Secret Lives of + Data. --- # Consensus Protocol -Nomad uses a [consensus protocol](http://en.wikipedia.org/wiki/Consensus_(computer_science)) -to provide [Consistency (as defined by CAP)](http://en.wikipedia.org/wiki/CAP_theorem). +Nomad uses a [consensus protocol](https://en.wikipedia.org/wiki/Consensus_(computer_science)) +to provide [Consistency (as defined by CAP)](https://en.wikipedia.org/wiki/CAP_theorem). The consensus protocol is based on ["Raft: In search of an Understandable Consensus Algorithm"](https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf). For a visual explanation of Raft, see [The Secret Lives of Data](http://thesecretlivesofdata.com/raft). ~> **Advanced Topic!** This page covers technical details of -the internals of Nomad. You don't need to know these details to effectively +the internals of Nomad. You do not need to know these details to effectively operate and use Nomad. These details are documented here for those who wish to learn about them without having to go spelunking through the source code. ## Raft Protocol Overview Raft is a consensus algorithm that is based on -[Paxos](http://en.wikipedia.org/wiki/Paxos_%28computer_science%29). Compared +[Paxos](https://en.wikipedia.org/wiki/Paxos_%28computer_science%29). Compared to Paxos, Raft is designed to have fewer states and a simpler, more understandable algorithm. There are a few key terms to know when discussing Raft: -* Log - The primary unit of work in a Raft system is a log entry. The problem +* **Log** - The primary unit of work in a Raft system is a log entry. The problem of consistency can be decomposed into a *replicated log*. A log is an ordered sequence of entries. We consider the log consistent if all members agree on the entries and their order. -* FSM - [Finite State Machine](http://en.wikipedia.org/wiki/Finite-state_machine). +* **FSM** - [Finite State Machine](https://en.wikipedia.org/wiki/Finite-state_machine). An FSM is a collection of finite states with transitions between them. As new logs are applied, the FSM is allowed to transition between states. Application of the same sequence of logs must result in the same state, meaning behavior must be deterministic. -* Peer set - The peer set is the set of all members participating in log replication. +* **Peer set** - The peer set is the set of all members participating in log replication. For Nomad's purposes, all server nodes are in the peer set of the local region. -* Quorum - A quorum is a majority of members from a peer set: for a set of size `n`, +* **Quorum** - A quorum is a majority of members from a peer set: for a set of size `n`, quorum requires at least `(n/2)+1` members. For example, if there are 5 members in the peer set, we would need 3 nodes to form a quorum. If a quorum of nodes is unavailable for any reason, the cluster becomes *unavailable* and no new logs can be committed. -* Committed Entry - An entry is considered *committed* when it is durably stored +* **Committed Entry** - An entry is considered *committed* when it is durably stored on a quorum of nodes. Once an entry is committed it can be applied. -* Leader - At any given time, the peer set elects a single node to be the leader. +* **Leader** - At any given time, the peer set elects a single node to be the leader. The leader is responsible for ingesting new log entries, replicating to followers, and managing when an entry is considered committed. diff --git a/website/source/docs/internals/gossip.html.md b/website/source/docs/internals/gossip.html.md index 240809939..8f9efe486 100644 --- a/website/source/docs/internals/gossip.html.md +++ b/website/source/docs/internals/gossip.html.md @@ -3,19 +3,20 @@ layout: "docs" page_title: "Gossip Protocol" sidebar_current: "docs-internals-gossip" description: |- - Nomad uses a gossip protocol to manage membership. All of this is provided through the use of the Serf library. + Nomad uses a gossip protocol to manage membership. All of this is provided + through the use of the Serf library. --- # Gossip Protocol -Nomad uses a [gossip protocol](http://en.wikipedia.org/wiki/Gossip_protocol) +Nomad uses a [gossip protocol](https://en.wikipedia.org/wiki/Gossip_protocol) to manage membership. This is provided through the use of the [Serf library](https://www.serfdom.io/). The gossip protocol used by Serf is based on -["SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol"](http://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf), +["SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol"](https://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf), with a few minor adaptations. There are more details about [Serf's protocol here](https://www.serfdom.io/docs/internals/gossip.html). ~> **Advanced Topic!** This page covers technical details of -the internals of Nomad. You don't need to know these details to effectively +the internals of Nomad. You do not need to know these details to effectively operate and use Nomad. These details are documented here for those who wish to learn about them without having to go spelunking through the source code. diff --git a/website/source/docs/internals/index.html.md b/website/source/docs/internals/index.html.md index 8db3cc7aa..bbaf136c3 100644 --- a/website/source/docs/internals/index.html.md +++ b/website/source/docs/internals/index.html.md @@ -13,5 +13,5 @@ details of how Nomad functions, its architecture and sub-systems. -> **Note:** Knowledge of Nomad internals is not required to use Nomad. If you aren't interested in the internals -of Nomad, you may safely skip this section. If you're operating Nomad, +of Nomad, you may safely skip this section. If you are operating Nomad, we recommend understanding the internals. diff --git a/website/source/docs/internals/scheduling.html.md b/website/source/docs/internals/scheduling.html.md index ca930d85e..c9f222471 100644 --- a/website/source/docs/internals/scheduling.html.md +++ b/website/source/docs/internals/scheduling.html.md @@ -13,18 +13,18 @@ from jobs to client machines. This process must respect the constraints as decla in the job, and optimize for resource utilization. This page documents the details of how scheduling works in Nomad to help both users and developers build a mental model. The design is heavily inspired by Google's -work on both [Omega: flexible, scalable schedulers for large compute clusters](http://research.google.com/pubs/pub41684.html) -and [Large-scale cluster management at Google with Borg](http://research.google.com/pubs/pub43438.html). +work on both [Omega: flexible, scalable schedulers for large compute clusters](https://research.google.com/pubs/pub41684.html) +and [Large-scale cluster management at Google with Borg](https://research.google.com/pubs/pub43438.html). ~> **Advanced Topic!** This page covers technical details -of Nomad. You don't need to understand these details to +of Nomad. You do not need to understand these details to effectively use Nomad. The details are documented here for those who wish to learn about them without having to go spelunking through the source code. # Scheduling in Nomad -![Data Model](/assets/images/nomad-nouns.png) +[![Nomad Data Model](/assets/images/nomad-data-model.png)](/assets/images/nomad-data-model.png) There are four primary "nouns" in Nomad, these are jobs, nodes, allocations, and evaluations. Jobs are submitted by users and represent a _desired state_. A job is a declarative description @@ -43,7 +43,7 @@ it with the desired state. This diagram shows the flow of an evaluation through Nomad: -![Evaluation Flow](/assets/images/eval-flow.png) +[![Nomad Evaluation Flow](/assets/images/nomad-evaluation-flow.png)](/assets/images/nomad-evaluation-flow.png) The lifecycle of an evaluation beings with an event causing the evaluation to be created. Evaluations are created in the `pending` state and are enqueued into the diff --git a/website/source/docs/internals/telemetry.html.md b/website/source/docs/internals/telemetry.html.md index ab6d543f1..8c2ee623b 100644 --- a/website/source/docs/internals/telemetry.html.md +++ b/website/source/docs/internals/telemetry.html.md @@ -19,7 +19,7 @@ it will dump the current telemetry information to the agent's `stderr`. This telemetry information can be used for debugging or otherwise getting a better view of what Nomad is doing. -Telemetry information can be streamed to both [statsite](http://github.com/armon/statsite) +Telemetry information can be streamed to both [statsite](https://github.com/armon/statsite) as well as statsd based on providing the appropriate configuration options. Below is sample output of a telemetry dump: diff --git a/website/source/downloads.html.erb b/website/source/downloads.html.erb index d646e0e44..637a8e87f 100644 --- a/website/source/downloads.html.erb +++ b/website/source/downloads.html.erb @@ -1,7 +1,7 @@ --- layout: "downloads" page_title: "Download Nomad" -sidebar_current: "downloads-vault" +sidebar_current: "downloads-nomad" description: |- Download Nomad --- @@ -15,7 +15,7 @@ description: |- Below are all available downloads for the latest version of Nomad (<%= latest_version %>). Please download the proper package for your operating system and architecture. You can find SHA256 checksums - for packages here. You can verify the SHA256 checksums using our PGP public key and the SHA256SUMs signature file. + for packages here. You can verify the SHA256 checksums using our PGP public key and the SHA256SUMs signature file.

diff --git a/website/source/intro/getting-started/acl.html.md b/website/source/intro/getting-started/acl.html.md deleted file mode 100644 index 9bc18b9a5..000000000 --- a/website/source/intro/getting-started/acl.html.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -layout: "intro" -page_title: "Access Control Policies" -sidebar_current: "gettingstarted-acl" -description: |- - Access control policies in Nomad control what a user can access. ---- - -# Access Control Policies (ACLs) - -Access control policies in Nomad control what a user can access. In -the last section, we learned about _authentication_. This section is -about _authorization_. - -Whereas for authentication Nomad has multiple options or backends that -can be enabled and used, the authorization or policies of Nomad are always -the same format. All authentication backends must map identities back to -the core policies that are configured with Nomad. - -When initializing Nomad, there is always one special policy created -that can't be removed: the "root" policy. This policy is a special policy -that gives superuser access to everything in Nomad. An identity mapped to -the root policy can do anything. - -## Policy Format - -Policies in Nomad are formatted with -[HCL](https://github.com/hashicorp/hcl). HCL is a human-readable configuration -format that is also JSON-compatible, so you can use JSON as well. An example -policy is shown below: - -```javascript -path "secret/*" { - policy = "write" -} - -path "secret/foo" { - policy = "read" -} -``` - -The policy format uses a prefix matching system on the API path -to determine access control. The most specific defined policy is used, -either an exact match or the longest-prefix glob match. Since everything -in Nomad must be accessed via the API, this gives strict control over every -aspect of Nomad, including mounting backends, authenticating, as well as secret access. - -In the policy above, a user could write any secret to `secret/`, except -to `secret/foo`, where only read access is allowed. Policies default to -deny, so any access to an unspecified path is not allowed. The policy -language changed slightly in Nomad 0.2, [see this page for details](/docs/concepts/policies.html). - -Save the above policy as `acl.hcl`. - -## Writing the Policy - -To write a policy, use the `vault policy-write` command: - -``` -$ vault policy-write secret acl.hcl -Policy 'secret' written. -``` - -You can see the policies that are available with `vault policies`, and you -can see the contents of a policy with `vault policies `. Only users with -root access can do this. - -## Testing the Policy - -To use the policy, let's create a token and assign it to that policy. -Make sure to save your root token somewhere so you can authenticate -back to a root user later. - -``` -$ vault token-create -policy="secret" -d97ef000-48cf-45d9-1907-3ea6ce298a29 - -$ vault auth d97ef000-48cf-45d9-1907-3ea6ce298a29 -Successfully authenticated! The policies that are associated -with this token are listed below: - -secret -``` - -You can now verify that you can write data to `secret/`, but only -read from `secret/foo`: - -``` -$ vault write secret/bar value=yes -Success! Data written to: secret/bar - -$ vault write secret/foo value=yes -Error writing data to secret/foo: Error making API request. - -URL: PUT http://127.0.0.1:8200/v1/secret/foo -Code: 400. Errors: - -* permission denied -``` - -You also don't have access to `sys` according to the policy, so commands -such as `vault mounts` will not work either. - -## Mapping Policies to Auth Backends - -Nomad is the single policy authority, unlike auth where you can mount -multiple backends. Any mounted auth backend must map identities to these -core policies. - -Use the `vault path-help` system with your auth backend to determine how the -mapping is done, since it is specific to each backend. For example, -with GitHub, it is done by team using the `map/teams/` path: - -``` -$ vault write auth/github/map/teams/default value=secret -Success! Data written to: auth/github/map/teams/default -``` - -For GitHub, the "default" team is the default policy set that everyone -is assigned to no matter what team they're on. - -Other auth backends use alternate, but likely similar mechanisms for -mapping policies to identity. - -## Next - -Policies are an important part of Nomad. While using the root token -is easiest to get up and running, you'll want to restrict access to -Nomad very quickly, and the policy system is the way to do this. - -The syntax and function of policies is easy to understand and work -with, and because auth backends all must map to the central policy system, -you only have to learn this policy system. - -Next, we'll cover how to [deploy Nomad](/intro/getting-started/deploy.html). diff --git a/website/source/intro/getting-started/apis.html.md b/website/source/intro/getting-started/apis.html.md index f2bac7499..6863125f8 100644 --- a/website/source/intro/getting-started/apis.html.md +++ b/website/source/intro/getting-started/apis.html.md @@ -1,203 +1,16 @@ --- layout: "intro" page_title: "Using the HTTP APIs with Authentication" -sidebar_current: "gettingstarted-apis" +sidebar_current: "getting-started-apis" description: |- Using the HTTP APIs for authentication and secret access. --- # Using the HTTP APIs with Authentication -All of Nomad's capabilities are accessible via the HTTP API in addition to the CLI. In fact, most calls from the CLI actually invoke the HTTP API. In some cases, Nomad features are not available via the CLI and can only be accessed via the HTTP API. +Many of Nomad's capabilities are accessible via the HTTP API in addition to the +CLI. -Once you have started the Nomad server, you can use `curl` or any other http client to make API calls. For example, if you started the Nomad server in [development mode](/docs/concepts/dev-server.html), you could validate the initialization status like this: - -``` -$ curl http://127.0.0.1:8200/v1/sys/init -``` - -This will return a JSON response: - -```javascript -{"initialized":true} -``` - -## Accessing Secrets via the REST APIs -Machines that need access to information stored in Nomad will most likely access Nomad via its REST API. For example, if a machine were using [app-id](/docs/auth/app-id.html) for authentication, the application would first authenticate to Nomad which would return a Nomad API token. The application would use that token for future communication with Nomad. - -For the purpose of this guide, we will use the following configuration which disables TLS and uses a file-based backend. You should never disable TLS in production, but it is okay for the purposes of this tutorial. - -```javascript -backend "file" { - path = "vault" -} - -listener "tcp" { - tls_disable = 1 -} -``` - -Save this file on disk and then start the Nomad server with this command: - -``` -$ vault server -config=/etc/vault.conf -``` - -At this point, we can use Nomad's API for all our interactions. For example, we can initialize Nomad like this: - -``` -$ curl \ - -X PUT \ - -d "{\"secret_shares\":1, \"secret_threshold\":1}" \ - http://localhost:8200/v1/sys/init -``` - -The response should be JSON and look something like this: - -```javascript -{ - "keys": ["69cf1c12a1f65dddd19472330b28cf4e95c657dfbe545877e5765d25d0592b16"], - "root_token": "0e2ede5a-6664-a49e-ca33-8f204d1cdb95" -} -``` - -This response contains our initial root token. It also includes the unseal key. You can use the uneal key to unseal the Nomad and use the root token perform other requests in Nomad that require authentication. - -To make this guide easy to copy-and-paste, we will be using the environment variable `$VAULT_TOKEN` to store the root token. You can export this Nomad token in your current shell like this: - -``` -$ export VAULT_TOKEN=0e2ede5a-6664-a49e-ca33-8f204d1cdb95 -``` - -Using the unseal key (not the root token) from above, you can unseal the Nomad via the HTTP API: - -``` -$ curl \ - -X PUT \ - -d '{"key": "69cf1c12a1f65dddd19472330b28cf4e95c657dfbe545877e5765d25d0592b16"}' \ - http://127.0.0.1:8200/v1/sys/unseal -``` - -Note that you should replace `69cf1c1...` with the generated key from your output. This will return a JSON response: - -```javascript -{ - "sealed": false, - "t": 1, - "n": 1, - "progress": 0 -} -``` - -Now we can enable an authentication backend such as [GitHub authentication](/docs/auth/github.html) or [App ID](/docs/auth/app-id.html). For the purposes of this guide, we will enable App ID authentication. - -We can enable an authentication backend with the following `curl` command: - -``` -$ curl \ - -X POST \ - -H "X-Nomad-Token:$VAULT_TOKEN" \ - -d '{"type":"app-id"}' \ - http://127.0.0.1:8200/v1/sys/auth/app-id -``` - -Notice that the request to the app-id endpoint needed an authentication token. In this case we are passing the root token generated when we started the Nomad server. We could also generate tokens using any other authentication mechanisms, but we will use the root token for simplicity. - -The last thing we need to do before using our App ID endpoint is writing the data to the store to associate an app id with a user id. For more information on this process, see the documentation on the [app-id auth backend](/docs/auth/app-id.html). - -First, we need to associate the application with a particular [ACL policy](/docs/concepts/policies.html) in Nomad. In the following command, we are going to associate the created tokens with the `root` policy. You would not want to do this in a real production scenario because the root policy allows complete read, write, and administrator access to Nomad. For a production application, you should create an ACL policy (which is also possible via the HTTP API), but is not covered in this guide for simplicity. - -``` -$ curl \ - -X POST \ - -H "X-Nomad-Token:$VAULT_TOKEN" \ - -d '{"value":"root", "display_name":"demo"}' \ - http://localhost:8200/v1/auth/app-id/map/app-id/152AEA38-85FB-47A8-9CBD-612D645BFACA -``` - -Note that `152AEA38-85FB-47A8-9CBD-612D645BFACA` is a randomly generated UUID. You can use any tool to generate a UUID, but make sure it is unique. - -Next we need to map the application to a particular "user". In Nomad, this is actually a particular application: - -``` -$ curl \ - -X POST \ - -H "X-Nomad-Token:$VAULT_TOKEN" \ - -d '{"value":"152AEA38-85FB-47A8-9CBD-612D645BFACA"}' \ - http://localhost:8200/v1/auth/app-id/map/user-id/5ADF8218-D7FB-4089-9E38-287465DBF37E -``` - -Now your app can identify itself via the app-id and user-id and get access to Nomad. The first step is to authenticate: - -``` -$ curl \ - -X POST \ - -d '{"app_id":"152AEA38-85FB-47A8-9CBD-612D645BFACA", "user_id": "5ADF8218-D7FB-4089-9E38-287465DBF37E"}' \ - "http://127.0.0.1:8200/v1/auth/app-id/login" -``` - -This will return a response that looks like the following: - -```javascript -{ - "lease_id": "", - "renewable": false, - "lease_duration": 0, - "data": null, - "auth": { - "client_token": "7a25c58b-9bad-5750-b579-edbb9f10a5ef", - "policies": ["root"], - "lease_duration": 0, - "renewable": false, - "metadata": { - "app-id": "sha1:1c0401b419280b0771d006bcdae683989086a00e", - "user-id": "sha1:4dbf74fce71648d54c42e28ad193253600853ca6" - } - } -} -``` - -The returned client token (`7a25c58b-9bad-5750-b579-edbb9f10a5ef`) can now be used to authenticate with Nomad. As you can see from the returned payload, the App ID backend does not currently support lease expiration or renewal. If you authenticate with backend that does support leases, your application will have to track expiration and handle renewal, but that is a topic for another guide. - -We can export this new token as our new `VAULT_TOKEN`: - -``` -$ export VAULT_TOKEN="7a25c58b-9bad-5750-b579-edbb9f10a5ef" -``` - -Be sure to replace this with the value returned from your API response. We can now use this token to authentication requests to Nomad: - -``` -$ curl \ - -X POST \ - -H "X-Nomad-Token:$VAULT_TOKEN" \ - -H 'Content-type: application/json' \ - -d '{"bar":"baz"}' \ - http://127.0.0.1:8200/v1/secret/foo -``` - -This will create a new secret named "foo" with the given JSON contents. We can read this value back with the same token: - -``` -$ curl \ - -H "X-Nomad-Token:$VAULT_TOKEN" \ - http://127.0.0.1:8200/v1/secret/foo -``` - -This should return a response like this: - -```javascript -{ - "lease_id": "secret/foo/cc529d06-36c8-be27-31f5-2390e1f6e2ae", - "renewable": false, - "lease_duration": 2592000, - "data": { - "bar": "baz" - }, - "auth": null -} -``` - -You can see the documentation on the [HTTP APIs](/docs/http/index.html) for more details on other available endpoints. +TODO: Document Nomad's HTTP API Congratulations! You now know all the basics to get started with Nomad. diff --git a/website/source/intro/getting-started/authentication.html.md b/website/source/intro/getting-started/authentication.html.md deleted file mode 100644 index 253152a85..000000000 --- a/website/source/intro/getting-started/authentication.html.md +++ /dev/null @@ -1,167 +0,0 @@ ---- -layout: "intro" -page_title: "Authentication" -sidebar_current: "gettingstarted-auth" -description: |- - Authentication to Nomad gives a user access to use Nomad. Nomad can authenticate using multiple methods. ---- - -# Authentication - -Now that we know how to use the basics of Nomad, it is important to understand -how to authenticate to Nomad itself. Up to this point, we haven't had to -authenticate because starting the Nomad server in dev mode automatically logs -us in as root. In practice, you'll almost always have to manually authenticate. - -On this page, we'll talk specifically about _authentication_. On the next -page, we talk about _authorization_. -Authentication is the mechanism of assigning an identity to a Nomad user. -The access control and permissions associated with an identity are -authorization, and will not be covered on this page. - -Nomad has pluggable authentication backends, making it easy to authenticate -with Nomad using whatever form works best for your organization. On this page -we'll use the token backend as well as the GitHub backend. - -## Tokens - -We'll first explain token authentication before going over any other -authentication backends. Token authentication is enabled by default in -Nomad and cannot be disabled. It is also what we've been using up to this -point. - -When you start a dev server with `vault server -dev`, it outputs your -_root token_. The root token is the initial access token to configure Nomad. -It has root privileges, so it can perform any operation within Nomad. -We'll cover how to limit privileges in the next section. - -You can create more tokens using `vault token-create`: - -``` -$ vault token-create -6c38f603-6441-2161-c543-ee15b7206563 -``` - -By default, this will create a child token of your current token that -inherits all the same access control policies. The "child" concept here -is important: tokens always have a parent, and when that parent token is -revoked, children can also be revoked all in one operation. This makes it -easy when removing access for a user, to remove access for all sub-tokens -that user created as well. - -After a token is created, you can revoke it with `vault token-revoke`: - -``` -$ vault token-revoke 6c38f603-6441-2161-c543-ee15b7206563 -Revocation successful. -``` - -In a previous section, we use the `vault revoke` command. This command -is only used for revoking _secrets_. For revoking _tokens_, the -`vault token-revoke` command must be used. - -To authenticate with a token, use the `vault auth` command: - -``` -$ vault auth d08e2bd5-ffb0-440d-6486-b8f650ec8c0c -Successfully authenticated! The policies that are associated -with this token are listed below: - -root -``` - -This authenticates with Nomad. It will verify your token and let you know -what access policies the token is associated with. If you want to test -`vault auth`, make sure you create a new token first. - -## Auth Backends - -In addition to tokens, other authentication backends can be enabled. -Authentication backends enable alternate methods of identifying with Nomad. -These identities are tied back to a set of access policies, just like tokens. - -Nomad supports other authentication backends in order to make authentication -easiest for your environment. For example, for desktop environments, -private key or GitHub based authentication may be easiest. For server -environments, some shared secret may be best. Auth backends give you -flexibility to choose what authentication you want to use. - -As an example, let's authenticate using GitHub. First, enable the -GitHub authentication backend: - -``` -$ vault auth-enable github -Successfully enabled 'github' at 'github'! -``` - -Auth backends are mounted, just like secret backends, except auth -backends are always prefixed with `auth/`. So the GitHub backend we just -mounted can be accessed at `auth/github`. You can use `vault path-help` to -learn more about it. - -With the backend enabled, we first have to configure it. For GitHub, -we tell it what organization users must part of, and map a team to a policy: - -``` -$ vault write auth/github/config organization=hashicorp -Success! Data written to: auth/github/config - -$ vault write auth/github/map/teams/default value=root -Success! Data written to: auth/github/map/teams/default -``` - -The above configured our GitHub backend to only accept users from the -"hashicorp" organization (you should fill in your own organization) -and to map any team to the "root" policy, which is the only policy we have -right now until the next section. - -With GitHub enabled, we can authenticate using `vault auth`: - -``` -$ vault auth -method=github token=e6919b17dd654f2b64e67b6369d61cddc0bcc7d5 -Successfully authenticated! The policies that are associated -with this token are listed below: - -root -``` - -Success! We've authenticated using GitHub. The "root" policy was associated -with my identity since we mapped that earlier. The value for "token" should be your own -[personal access token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/). - -You can revoke authentication from any authentication backend using -`vault token-revoke` as well, which can revoke any path prefix. For -example, to revoke all GitHub tokens, you could run the following. -**Don't run this unless you have access to another root token or you'll -get locked out.** - -``` -$ vault token-revoke -mode=path auth/github -``` - -When you're done, you can disable authentication backends with -`vault auth-disable`. This will immediately invalidate all authenticated -users from this backend. - -``` -$ vault auth-disable github -Disabled auth provider at path 'github'! -``` - -If you ran the above, you'll probably find you can't access your Nomad -anymore unless you have another root token, since it invalidated your -own session since we authenticated with GitHub above. Since we're still -operating in development mode, just restart the dev server to fix this. - -## Next - -In this page you learned about how Nomad authenticates users. You learned -about the built-in token system as well as enabling other authentication -backends. At this point you know how Nomad assigns an _identity_ to -a user. - -The multiple authentication backends Nomad provides let you choose the -most appropriate authentication mechanism for your organization. - -In this next section, we'll learn about -[access control policies](/intro/getting-started/acl.html). diff --git a/website/source/intro/getting-started/deploy.html.md b/website/source/intro/getting-started/deploy.html.md deleted file mode 100644 index bd1b8f398..000000000 --- a/website/source/intro/getting-started/deploy.html.md +++ /dev/null @@ -1,222 +0,0 @@ ---- -layout: "intro" -page_title: "Deploy Nomad" -sidebar_current: "gettingstarted-deploy" -description: |- - Learn how to deploy Nomad into production, how to initialize it, configure it, etc. ---- - -# Deploy Nomad - -Up to this point, we've been working with the dev server, which -automatically authenticated us, setup in-memory storage, etc. Now that -you know the basics of Nomad, it is important to learn how to deploy -Nomad into a real environment. - -On this page, we'll cover how to configure Nomad, start Nomad, the -seal/unseal process, and scaling Nomad. - -## Configuring Nomad - -Nomad is configured using [HCL](https://github.com/hashicorp/hcl) files. -As a reminder, these files are also JSON-compatible. The configuration -file for Nomad is relatively simple. An example is shown below: - -```javascript -backend "consul" { - address = "127.0.0.1:8500" - path = "vault" -} - -listener "tcp" { - address = "127.0.0.1:8200" - tls_disable = 1 -} -``` - -Within the configuration file, there are two primary configurations: - - * `backend` - This is the physical backend that Nomad uses for - storage. Up to this point the dev server has used "inmem" (in memory), - but in the example above we're using [Consul](http://www.consul.io), - a much more production-ready backend. - - * `listener` - One or more listeners determine how Nomad listens for - API requests. In the example above we're listening on localhost port - 8200 without TLS. - -For now, copy and paste the configuration above to `example.hcl`. It will -configure Nomad to expect an instance of Consul running locally. - -Starting a local Consul instance takes only a few minutes. Just follow the -[Consul Getting Started Guide](https://www.consul.io/intro/getting-started/install.html) -up to the point where you have installed Consul and started it with this command: - -```shell -$ consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul -``` - -## Starting the Server - -With the configuration in place, starting the server is simple, as -shown below. Modify the `-config` flag to point to the proper path -where you saved the configuration above. - -``` -$ vault server -config=example.hcl -==> Nomad server configuration: - - Log Level: info - Backend: consul - Listener 1: tcp (addr: "127.0.0.1:8200", tls: "disabled") - -==> Nomad server started! Log data will stream in below: -``` - -Nomad outputs some information about its configuration, and then blocks. -This process should be run using a resource manager such as systemd or -upstart. - -You'll notice that you can't execute any commands. We don't have any -auth information! When you first setup a Nomad server, you have to start -by _initializing_ it. - -On Linux, Nomad may fail to start with the following error: - -```shell -$ vault server -config=example.hcl -Error initializing core: Failed to lock memory: cannot allocate memory - -This usually means that the mlock syscall is not available. -Nomad uses mlock to prevent memory from being swapped to -disk. This requires root privileges as well as a machine -that supports mlock. Please enable mlock on your system or -disable Nomad from using it. To disable Nomad from using it, -set the `disable_mlock` configuration option in your configuration -file. -``` - -For guidance on dealing with this issue, see the discussion of -`disable_mlock` in [Server Configuration](/docs/config/index.html). - -## Initializing the Nomad - -Initialization is the process of first configuring the Nomad. This -only happens once when the server is started against a new backend that -has never been used with Nomad before. - -During initialization, the encryption keys are generated, unseal keys -are created, and the initial root token is setup. To initialize Nomad -use `vault init`. This is an _unauthenticated_ request, but it only works -on brand new Nomads with no data: - -``` -$ vault init -Key 1: 427cd2c310be3b84fe69372e683a790e01 -Key 2: 0e2b8f3555b42a232f7ace6fe0e68eaf02 -Key 3: 37837e5559b322d0585a6e411614695403 -Key 4: 8dd72fd7d1af254de5f82d1270fd87ab04 -Key 5: b47fdeb7dda82dbe92d88d3c860f605005 -Initial Root Token: eaf5cc32-b48f-7785-5c94-90b5ce300e9b - -Nomad initialized with 5 keys and a key threshold of 3! - -Please securely distribute the above keys. Whenever a Nomad server -is started, it must be unsealed with 3 (the threshold) -of the keys above (any of the keys, as long as the total number equals -the threshold). - -Nomad does not store the original master key. If you lose the keys -above such that you no longer have the minimum number (the -threshold), then your Nomad will not be able to be unsealed. -``` - -Initialization outputs two incredibly important pieces of information: -the _unseal keys_ and the _initial root token_. This is the -**only time ever** that all of this data is known by Nomad, and also the -only time that the unseal keys should ever be so close together. - -For the purpose of this getting started guide, save all these keys -somewhere, and continue. In a real deployment scenario, you would never -save these keys together. - -## Seal/Unseal - -Every initialized Nomad server starts in the _sealed_ state. From -the configuration, Nomad can access the physical storage, but it can't -read any of it because it doesn't know how to decrypt it. The process -of teaching Nomad how to decrypt the data is known as _unsealing_ the -Nomad. - -Unsealing has to happen every time Nomad starts. It can be done via -the API and via the command line. To unseal the Nomad, you -must have the _threshold_ number of unseal keys. In the output above, -notice that the "key threshold" is 3. This means that to unseal -the Nomad, you need 3 of the 5 keys that were generated. - --> **Note:** Nomad does not store any of the unseal key shards. Nomad -uses an algorithm known as -[Shamir's Secret Sharing](http://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing) -to split the master key into shards. Only with the threshold number of keys -can it be reconstructed and your data finally accessed. - -Begin unsealing the Nomad with `vault unseal`: - -``` -$ vault unseal -Key (will be hidden): -Sealed: true -Key Shares: 5 -Key Threshold: 3 -Unseal Progress: 1 -``` - -After pasting in a valid key and confirming, you'll see that the Nomad -is still sealed, but progress is made. Nomad knows it has 1 key out of 3. -Due to the nature of the algorithm, Nomad doesn't know if it has the -_correct_ key until the threshold is reached. - -Also notice that the unseal process is stateful. You can go to another -computer, use `vault unseal`, and as long as it's pointing to the same server, -that other computer can continue the unseal process. This is incredibly -important to the design of the unseal process: multiple people with multiple -keys are required to unseal the Nomad. The Nomad can be unsealed from -multiple computers and the keys should never be together. A single malicious -operator does not have enough keys to be malicious. - -Continue with `vault unseal` to complete unsealing the Nomad. Note that -all 3 keys must be different, but they can be any other keys. As long as -they're correct, you should soon see output like this: - -``` -$ vault unseal -Key (will be hidden): -Sealed: false -Key Shares: 5 -Key Threshold: 3 -Unseal Progress: 0 -``` - -The `Sealed: false` means the Nomad is unsealed! - -Feel free to play around with entering invalid keys, keys in different -orders, etc. in order to understand the unseal process. It is very important. -Once you're ready to move on, use `vault auth` to authenticate with -the root token. - -As a root user, you can reseal the Nomad with `vault seal`. A single -operator is allowed to do this. This lets a single operator lock down -the Nomad in an emergency without consulting other operators. - -When the Nomad is sealed again, it clears all of its state (including -the encryption key) from memory. The Nomad is secure and locked down -from access. - -## Next - -You now know how to configure, initialize, and unseal/seal Nomad. -This is the basic knowledge necessary to deploy Nomad into a real -environment. Once the Nomad is unsealed, you access it as you have -throughout this getting started guide (which worked with an unsealed Nomad). - -Next, we have a [short tutorial](/intro/getting-started/apis.html) on using the HTTP APIs to authenticate and access secrets. diff --git a/website/source/intro/getting-started/dev-server.html.md b/website/source/intro/getting-started/dev-server.html.md deleted file mode 100644 index b8253f999..000000000 --- a/website/source/intro/getting-started/dev-server.html.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -layout: "intro" -page_title: "Starting the Server" -sidebar_current: "gettingstarted-devserver" -description: |- - After installing Nomad, the next step is to start the server. ---- - -# Starting the Nomad Server - -With Nomad installed, the next step is to start a Nomad server. - -Nomad operates as a client/server application. The Nomad server is the -only piece of the Nomad architecture that interacts with the data -storage and backends. All operations done via the Nomad CLI interact -with the server over a TLS connection. - -In this page, we'll start and interact with the Nomad server to understand -how the server is started, and understanding the seal/unseal process. - -## Starting the Dev Server - -To start, we're going to start the Nomad _dev server_. The dev server -is a built-in flag to start a pre-configured server that is not very -secure but useful for playing with Nomad locally. Later in the getting -started guide we'll configure and start a real server. - -To start the Nomad dev server, run `vault server -dev`: - -``` -$ vault server -dev -WARNING: Dev mode is enabled! - -In this mode, Nomad is completely in-memory and unsealed. -Nomad is configured to only have a single unseal key. The root -token has already been authenticated with the CLI, so you can -immediately begin using the Nomad CLI. - -The only step you need to take is to set the following -environment variable since Nomad will be talking without TLS: - - export VAULT_ADDR='http://127.0.0.1:8200' - -The unseal key and root token are reproduced below in case you -want to seal/unseal the Nomad or play with authentication. - -Unseal Key: 2252546b1a8551e8411502501719c4b3 -Root Token: 79bd8011-af5a-f147-557e-c58be4fedf6c - -==> Nomad server configuration: - - Log Level: info - Backend: inmem - Listener 1: tcp (addr: "127.0.0.1:8200", tls: "disabled") - -... -``` - -You should see output similar to that above. As you can see, when you -start a dev server, Nomad warns you loudly. The dev server stores all -its data in-memory (but still encrypted), listens on localhost without TLS, and -automatically unseals and shows you the unseal key and root access key. -We'll go over what all this means shortly. - -The important thing about the dev server is that it is meant for -development only. **Do not run the dev server in production.** Even if it -was run in production, it wouldn't be very useful since it stores data in-memory -and every restart would clear all your secrets. - -With the dev server running, do the following three things before anything -else: - - 1. Copy and run the `export VAULT_ADDR ...` command from your terminal - output. This will configure the Nomad client to talk to our dev server. - - 2. Save the unseal key somewhere. Don't worry about _how_ to save this - securely. For now, just save it anywhere. - - 3. Do the same as step 2, but with the root token. We'll use this later. - -## Verify the Server is Running - -Verify the server is running by running `vault status`. This should -succeed and exit with exit code 0. If you see an error about opening -a connection, make sure you copied and executed the `export VAULT_ADDR...` -command from above properly. - -If it ran successful, the output should look like below: - -``` -$ vault status -Sealed: false -Key Shares: 1 -Key Threshold: 1 -Unseal Progress: 0 - -High-Availability Enabled: false -``` - -If the output looks different, especially if the numbers are different -or the Nomad is sealed, then restart the dev server and try again. The -only reason these would ever be different is if you're running a dev -server from going through this guide previously. - -We'll cover what this output means later in the guide. - -## Next - -Congratulations! You've started your first Nomad server. We haven't stored -any secrets yet, but we'll do that in the next section. - -Next, we're going to -[read and write our first secrets](/intro/getting-started/first-secret.html). diff --git a/website/source/intro/getting-started/dynamic-secrets.html.md b/website/source/intro/getting-started/dynamic-secrets.html.md deleted file mode 100644 index ede1addd1..000000000 --- a/website/source/intro/getting-started/dynamic-secrets.html.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -layout: "intro" -page_title: "Dynamic Secrets" -sidebar_current: "gettingstarted-dynamicsecrets" -description: |- - On this page we introduce dynamic secrets by showing you how to create AWS access keys with Nomad. ---- - -# Dynamic Secrets - -Now that we've written basic secrets to Nomad and we have an understanding -of the mount system, we're going to move on to the next core feature of -Nomad: _dynamic secrets_. - -Dynamic secrets are secrets that are generated when they're accessed, -and aren't statically written like we did in -[Your First Secret](/intro/getting-started/first-secret.html). -On this page, we'll use the built-in AWS secret backend to dynamically -generate AWS access keys. - -The power of dynamic secrets is that they simply don't exist before -they're read, so there is no risk of someone stealing them or another -client using the same secrets. And because Nomad has built-in revocation -mechanisms (covered later), the dynamic secret can be revoked right after -use, minimizing the amount of time the secret existed. - --> **Note:** Before starting this page, please register for an -[AWS account](http://aws.amazon.com). We won't be using any features that -cost money, so you shouldn't be charged for anything. However, we're not -responsible for any charges you may incur. - -## Mounting the AWS Backend - -Let's generate our first dynamic secret. We'll use the AWS backend to -dynamically generate an AWS access key pair. First, mount the AWS backend: - -``` -$ vault mount aws -Successfully mounted 'aws' at 'aws'! -``` - -The AWS backend is now mounted at `aws/`. As we covered in a previous -section: different secret backends allow for different behavior, and in this -case the AWS backend is a dynamic backend for generating AWS access credentials. - -## Configuring the AWS Backend - -With the AWS backend mounted, the first step is to configure it with -the AWS credentials that will be used to create the other credentials. -For now, use the root keys for your AWS account. - -To configure the backend, we use `vault write` to a special path -`aws/config/root`: - -``` -$ vault write aws/config/root \ - access_key=AKIAI4SGLQPBX6CSENIQ \ - secret_key=z1Pdn06b3TnpG+9Gwj3ppPSOlAsu08Qw99PUW+eB -Success! Data written to: aws/config/root -``` - -Remember that secret backends can behave anyway they want when -reading/writing a path, so this path stores this configuration for -later. Notice you can't read it back: - -``` -$ vault read aws/config/root -Error reading aws/config/root: Error making API request. - -URL: GET http://127.0.0.1:8200/v1/aws/config/root -Code: 500. Errors: - -* unsupported operation -``` - -To help keep the credentials secure, the AWS backend doesn't let you -read them back even if you're using a root credential. - -## Creating a Role - -The next step is to configure the AWS backend with an IAM policy. -IAM is the system AWS uses for creating new credentials with limited -API permissions. - -The AWS backend requires an IAM policy to associate created credentials -with. For this example, we'll write just one policy, but you can associate -many policies with the backend. Save a file named "policy.json" with the following contents: - -```javascript -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "Stmt1426528957000", - "Effect": "Allow", - "Action": [ - "ec2:*" - ], - "Resource": [ - "*" - ] - } - ] -} -``` - -This is a basic IAM policy that lets the user perform any action within -Amazon EC2. With the policy saved, write it to Nomad and create a new role: - -``` -$ vault write aws/roles/deploy policy=@policy.json -Success! Data written to: aws/roles/deploy -``` - -Again, we're using a special path here `aws/roles/` to write -an IAM policy to Nomad. We also used the special syntax `@filename` with -`vault write` to write the contents of a file. - -## Generating the Secret - -Now that we've configured the AWS backend and created a role, we can now -request an access key pair for that role. To do so, just read the -special path `aws/creds/` where `NAME` is the role name: - -``` -$ vault read aws/creds/deploy -Key Value -lease_id aws/creds/deploy/0d042c53-aa8a-7ce7-9dfd-310351c465e5 -access_key AKIAJFN42DVCQWDHQYHQ -secret_key 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 -``` - -Success! The access and secret key can now be used to perform any EC2 -operations within AWS. You can verify they work, if you want. Also notice -that these keys are new, they're not the keys you entered earlier. - -The `lease_id` above is a special ID used for Nomad for renewal, -revocation, etc. Copy and save your Lease ID now. - -## Revoking the Secret - -Let's complete the loop and revoke this secret now, purging it from -existence. Once the secret is revoked, the access keys will no longer -work. - -To revoke the secret, use `vault revoke` with the lease ID that was -outputted from `vault read` when you ran it: - -``` -$ vault revoke aws/creds/deploy/0d042c53-aa8a-7ce7-9dfd-310351c465e5 -Key revoked with ID 'aws/creds/deploy/0d042c53-aa8a-7ce7-9dfd-310351c465e5'. -``` - -Done! If you look at your AWS account, you'll notice that no IAM users -exist. If you try to use the access keys that were generated, you'll -find that they no longer work. - -With such easy dynamic creation and revocation, you can hopefully begin -to see how easy it is to work with dynamic secrets and ensure they only -exist for the duration that they're needed. - -## Next - -On this page we experienced our first dynamic secret, and we also saw -the revocation system in action. Dynamic secrets are incredibly powerful. -As time goes on, we expect that more systems will support some sort of -API to create access credentials, and Nomad will be ready to get the -most value out of this practice. - -Before going further, we're going to take a quick detour to learn -about the -[built-in help system](/intro/getting-started/help.html). diff --git a/website/source/intro/getting-started/first-secret.html.md b/website/source/intro/getting-started/first-secret.html.md deleted file mode 100644 index 30b2f6d47..000000000 --- a/website/source/intro/getting-started/first-secret.html.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -layout: "intro" -page_title: "Your First Secret" -sidebar_current: "gettingstarted-firstsecret" -description: |- - With the Nomad server running, let's read and write our first secret. ---- - -# Your First Secret - -Now that the dev server is up and running, let's get straight to it and -read and write our first secret. - -One of the core features of Nomad is the ability to read and write -arbitrary secrets securely. On this page, we'll do this using the CLI, -but there is also a complete -[HTTP API](/docs/http/index.html) -that can be used to programmatically do anything with Nomad. - -Secrets written to Nomad are encrypted and then written to the backend -storage. For our dev server, backend storage is in-memory, but in production -this would more likely be on disk or in [Consul](https://www.consul.io). -Nomad encrypts the value before it is ever handed to the storage driver. -The backend storage mechanism _never_ sees the unencrypted value and doesn't -have the means necessary to decrypt it without Nomad. - -## Writing a Secret - -Let's start by writing a secret. This is done very simply with the -`vault write` command, as shown below: - -``` -$ vault write secret/hello value=world -Success! Data written to: secret/hello -``` - -This writes the pair `value=world` to the path `secret/hello`. We'll -cover paths in more detail later, but for now it is important that the -path is prefixed with `secret/`, otherwise this example won't work. The -`secret/` prefix is where arbitrary secrets can be read and written. - -You can even write multiple pieces of data, if you want: - -``` -$ vault write secret/hello value=world excited=yes -Success! Data written to: secret/hello -``` - -`vault write` is a very powerful command. In addition to writing data -directly from the command-line, it can read values and keypairs from -stdin as well as files. For more information, see the -[vault write documentation](/docs/commands/read-write.html). - -~> **Warning:** The documentation uses the "key=value" based entry -throughout, but it is more secure to use files if possible. Sending -data via the CLI is often logged in shell history. For real secrets, -please use files. See the link above for more information. - -## Reading a Secret - -As you might expect, secrets can be read with `vault read`: - -``` -$ vault read secret/hello -Key Value -lease_id secret/hello/d57fe039-1eef-9f4a-f3c9-63e2b29002b8 -lease_duration 2592000 -excited yes -value world -``` - -As you can see, the values we wrote are given back to us. Nomad read -the data from storage and decrypted it. -The output format is purposefully whitespace separated to make it easy -to pipe into a tool like `awk`. - -In addition to the tabular format, if you're working with machines or -a tool like `jq`, you can output the data in JSON format: - -``` -$ vault read -format=json secret/hello -{ - "lease_id": "secret/hello/25f33857-15ed-b62b-dac6-4b29bb8e8bef", - "lease_duration": 2592000, - "renewable": false, - "data": { - "excited": "yes", - "value": "world" - } -} -``` - -This contains some extra information, but you can see our data mirrored -here as well. The JSON output is very useful for scripts. For example below -we use the `jq` tool to extract the "excited" value: - -``` -$ vault read -format=json secret/hello | jq -r .data.excited -yes -``` - -## Deleting a Secret - -Now that we've learned how to read and write a secret, let's go ahead -and delete it. We can do this with `vault delete`: - -``` -$ vault delete secret/hello -Success! Deleted 'secret/hello' -``` - -## Next - -In this section we learned how to use the powerful CRUD features of -Nomad to store arbitrary secrets. On its own this is already a useful -but basic feature. - -Next, we'll learn the basics about [secret backends](/intro/getting-started/secret-backends.html). diff --git a/website/source/intro/getting-started/help.html.md b/website/source/intro/getting-started/help.html.md deleted file mode 100644 index 827886b3d..000000000 --- a/website/source/intro/getting-started/help.html.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -layout: "intro" -page_title: "Built-in Help" -sidebar_current: "gettingstarted-help" -description: |- - Nomad has a built-in help system to learn about the available paths in Nomad and how to use them. ---- - -# Built-in Help - -You've now worked with `vault write` and `vault read` for multiple paths: -generic secret backend with `secret/` and dynamic AWS credentials with the -AWS backend provider at `aws/`. In both cases, the usage of read/write and -the paths to use differed. AWS in particular had special paths like -`aws/config`. - -Instead of having to memorize or reference documentation constantly -to determine what paths to use, we built a help system directly into -Nomad. This help system can be accessed via the API or the command-line and -generates human-readable help for any mounted backend. - -On this page, we'll learn how to use this help system. It is an invaluable -tool as you continue to work with Nomad. - -## Backend Overview - -For this, we'll assume you have the AWS backend mounted. If not, mount -it with `vault mount aws`. Even if you don't have an AWS account, you -can still mount the AWS backend. - -With the backend mounted, let's learn about it with `vault path-help`: - -``` -$ vault path-help aws -## DESCRIPTION - -The AWS backend dynamically generates AWS access keys for a set of -IAM policies. The AWS access keys have a configurable lease set and -are automatically revoked at the end of the lease. - -After mounting this backend, credentials to generate IAM keys must -be configured with the "root" path and policies must be written using -the "roles/" endpoints before any access keys can be generated. - -## PATHS - -The following paths are supported by this backend. To view help for -any of the paths below, use the help command with any route matching -the path pattern. Note that depending on the policy of your auth token, -you may or may not be able to access certain paths. - - ^config/lease$ - Configure the default lease information for generated credentials. - - ^config/root$ - Configure the root credentials that are used to manage IAM. - - ^creds/(?P\w+)$ - Generate an access key pair for a specific role. - - ^roles/(?P\w+)$ - Read and write IAM policies that access keys can be made for. -``` - -The `vault path-help` command takes a path. By specifying the root path for -a mount, it will give us the overview of that mount. Notice how the help -not only contains a description, but also the exact regular expressions -used to match routes for this backend along with a brief description -of what the route is for. - -## Path Help - -After seeing the overview, we can continue to dive deeper by getting -help for an individual path. For this, just use `vault path-help` with a path -that would match the regular expression for that path. Note that the path -doesn't need to actually _work_. For example, we'll get the help below -for accessing `aws/creds/operator`, even though we never created the `operator` -role: - -``` -$ vault path-help aws/creds/operator -Request: creds/operator -Matching Route: ^creds/(?P\w+)$ - -Generate an access key pair for a specific role. - -## PARAMETERS - - name (string) - Name of the role - -## DESCRIPTION - -This path will generate a new, never before used key pair for -accessing AWS. The IAM policy used to back this key pair will be -the "name" parameter. For example, if this backend is mounted at "aws", -then "aws/creds/deploy" would generate access keys for the "deploy" role. - -The access keys will have a lease associated with them. The access keys -can be revoked by using the lease ID. -``` - -Within a path, we're given the parameters that this path requires. -Some parameters come from the route itself. In this case, the "name" -parameter is a named capture from the route regular expression. - -There is also a description of what that path does. - -Go ahead and explore more paths! Mount other backends, traverse their -help systems and learn about what they do. For example, learn about the -generic `secret/` path. - -## Next - -The help system may not be the most exciting feature of Nomad, but it -is indispensable in day-to-day usage of Nomad. The help system lets you -learn about how to use any backend within Nomad without leaving the command -line. - -Next, we'll learn about -[authentication](/intro/getting-started/authentication.html). diff --git a/website/source/intro/getting-started/install.html.md b/website/source/intro/getting-started/install.html.md index c865fae6d..cbc71a344 100644 --- a/website/source/intro/getting-started/install.html.md +++ b/website/source/intro/getting-started/install.html.md @@ -1,7 +1,7 @@ --- layout: "intro" page_title: "Install Nomad" -sidebar_current: "gettingstarted-install" +sidebar_current: "getting-started-install" description: |- The first step to using Nomad is to get it installed. --- @@ -21,59 +21,35 @@ To install Nomad, find the [appropriate package](/downloads.html) for your system and download it. Nomad is packaged as a zip archive. After downloading Nomad, unzip the package. Nomad runs as a single binary -named `vault`. Any other files in the package can be safely removed and +named `nomad`. Any other files in the package can be safely removed and Nomad will still function. -The final step is to make sure that `vault` is available on the PATH. -See [this page](http://stackoverflow.com/questions/14637979/how-to-permanently-set-path-on-linux) +The final step is to make sure that `nomad` is available on the PATH. +See [this page](https://stackoverflow.com/questions/14637979/how-to-permanently-set-path-on-linux) for instructions on setting the PATH on Linux and Mac. -[This page](http://stackoverflow.com/questions/1618280/where-can-i-set-path-to-make-exe-on-windows) +[This page](https://stackoverflow.com/questions/1618280/where-can-i-set-path-to-make-exe-on-windows) contains instructions for setting the PATH on Windows. ## Verifying the Installation After installing Nomad, verify the installation worked by opening a new -terminal session and checking that `vault` is available. By executing -`vault`, you should see help output similar to the following: +terminal session and checking that `nomad` is available. By executing +`nomad`, you should see help output similar to the following: ``` -$ vault -usage: vault [-version] [-help] [args] +$ nomad +usage: nomad [--version] [--help] [] -Common commands: - delete Delete operation on secrets in Nomad - path-help Look up the help for a path - read Read data or secrets from Nomad - renew Renew the lease of a secret - revoke Revoke a secret. - server Start a Nomad server - status Outputs status of whether Nomad is sealed and if HA mode is enabled - write Write secrets or configuration into Nomad - -All other commands: - audit-disable Disable an audit backend - audit-enable Enable an audit backend - audit-list Lists enabled audit backends in Nomad - auth Prints information about how to authenticate with Nomad - auth-disable Disable an auth provider - auth-enable Enable a new auth provider - init Initialize a new Nomad server - key-status Provides information about the active encryption key - mount Mount a logical backend - mounts Lists mounted backends in Nomad - policies List the policies on the server - policy-delete Delete a policy from the server - policy-write Write a policy to the server - rekey Rekeys Nomad to generate new unseal keys - remount Remount a secret backend to a new path - rotate Rotates the backend encryption key used to persist data - seal Seals the vault server - token-create Create a new auth token - token-renew Renew an auth token - token-revoke Revoke one or more auth tokens - unmount Unmount a secret backend - unseal Unseals the vault server - version Prints the Nomad version +Available commands are: + agent Runs a Nomad agent + agent-force-leave Force a member into the 'left' state + agent-info Display status information about the local agent + agent-join Join server nodes together + agent-members Display a list of known members and their status + node-drain Toggle drain mode on a given node + node-status Display status information about nodes + status Display status information about jobs + version Prints the Nomad version ``` If you get an error that Nomad could not be found, then your PATH environment diff --git a/website/source/intro/getting-started/next-steps.html.md b/website/source/intro/getting-started/next-steps.html.md index 6d7d57b1d..747b98030 100644 --- a/website/source/intro/getting-started/next-steps.html.md +++ b/website/source/intro/getting-started/next-steps.html.md @@ -1,20 +1,19 @@ --- layout: "intro" page_title: "Next Steps" -sidebar_current: "gettingstarted-nextsteps" +sidebar_current: "getting-started-nextsteps" description: |- After completing the getting started guide, learn about what to do next with Nomad. --- # Next Steps -That concludes the getting started guide for Nomad. Hopefully you're +That concludes the getting started guide for Nomad. Hopefully you are excited about the possibilities of Nomad and ready to put this knowledge to use to improve your environment. We've covered the basics of all the core features of Nomad in this guide. -Due to the importance of securing secrets, we recommend reading the following -as next steps. +We recommend reading the following as next steps. * [Documentation](/docs/index.html) - The documentation is an in-depth reference guide to all the features of Nomad. diff --git a/website/source/intro/getting-started/running.md b/website/source/intro/getting-started/running.md new file mode 100644 index 000000000..478355857 --- /dev/null +++ b/website/source/intro/getting-started/running.md @@ -0,0 +1,18 @@ +--- +layout: "intro" +page_title: "Running Nomad" +sidebar_current: "getting-started-running" +description: |- + Learn how to deploy Nomad into production, how to initialize it, configure it, etc. +--- + +# Running Nomad +This section will detail how to run Nomad on client machines. It should include +a sample upstart script and stuff + +## Next + +TODO: Fill in text here. + +Next, we have a [short tutorial](/intro/getting-started/apis.html) on using +Nomad's HTTP APIs. diff --git a/website/source/intro/getting-started/secret-backends.html.md b/website/source/intro/getting-started/secret-backends.html.md deleted file mode 100644 index 4516fc1e8..000000000 --- a/website/source/intro/getting-started/secret-backends.html.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -layout: "intro" -page_title: "Secret Backends" -sidebar_current: "gettingstarted-secretbackends" -description: |- - Secret backends are what create, read, update, and delete secrets. ---- - -# Secret Backends - -Previously, we saw how to read and write arbitrary secrets to Nomad. -To do this, we used the `secret/` prefix. This prefix specifies the -_secret backend_ to use, and Nomad defaults to mounting the _generic_ -backend to `secret/`. The generic backend reads and writes raw data to -the backend storage. - -Nomad supports other backends in addition to "generic", and this feature -in particular is what makes Nomad unique. For example, the "aws" backend -generates AWS access keys dynamically, on demand. Another example -- -this type of backend does not yet exist -- is a backend that -reads and writes data directly to an -[HSM](http://en.wikipedia.org/wiki/Hardware_security_module). -As Nomad matures, more and more backends will be added. - -To represent backends, Nomad behaves much like a filesystem: backends -are _mounted_ at specific paths. For example, the "generic" backend is -_mounted_ at the `secret/` prefix. - -On this page, we'll learn about the mount system and the operations -that can be performed with them. We do this as prerequisite knowledge -to the next page, where we'll create dynamic secrets. - -## Mount a Backend - -To start, let's mount another "generic" backend. Just like a normal -filesystem, Nomad can mount a backend multiple times at different -mount points. This is useful if you want different access control policies -(covered later) or configurations for different paths. - -To mount the backend: - -``` -$ vault mount generic -Successfully mounted 'generic' at 'generic'! -``` - -By default, the mount point will be the same name as the backend. This -is because 99% of the time, you don't want to customize this mount point. -In this example, we mounted the "generic" backend at `generic/`. - -You can inspect mounts using `vault mounts`: - -``` -$ vault mounts -Path Type Description -generic/ generic -secret/ generic generic secret storage -sys/ system system endpoints used for control, policy and debugging -``` - -You can see the `generic/` path we just mounted, as well as the built-in -secret path. You can also see the `sys/` path. We won't cover this in the -getting started guide, but this mount point is used to interact with -the Nomad core system. - -Spend some time reading and writing secrets to the new mount point to -convince yourself it works. As a bonus, write to the `secret/` endpoint -and observe that those values are unavailable via `generic/`: they share the -same backend, but do not share any data. In addition to this, backends -(of the same type or otherwise) _cannot_ access the data of other backends; -they can only access data within their mount point. - -## Unmount a Backend - -Once you're sufficiently convinced mounts behave as you expect, you can -unmount it. When a backend is unmounted, all of its secrets are revoked -and its data is deleted. If either of these operations fail, the backend -remains mounted. - -``` -$ vault unmount generic/ -Successfully unmounted 'generic/'! -``` - -In addition to unmounting, you can _remount_ a backend. Remounting a -backend changes its mount point. This is still a disruptive command: the -stored data is retained, but all secrets are revoked since secrets are -closely tied to their mount paths. - -## What is a Secret Backend? - -Now that you've mounted and unmounted a backend, you might wonder: -"what is a secret backend? what is the point of this mounting system?" - -Nomad behaves a lot like a [virtual filesystem](http://en.wikipedia.org/wiki/Virtual_file_system). -The read/write/delete operations are forwarded to the backend, and the -backend can choose to react to these operations however it wishes. -For example, the "generic" backend simply passes this through to the -storage backend (after encrypting data first). - -However, the "aws" backend (which you'll see soon), will read/write IAM -policies and access tokens. So, while you might do a `vault read aws/deploy`, -this isn't reading from any _physical_ path "aws/deploy". Instead, the AWS -backend is dynamically generating an access key based on the "deploy" policy. - -This abstraction is incredibly powerful. It lets Nomad interface directly -with physical systems such as the backend as well as things such as SQL -databases, HSMs, etc. But in addition to these physical systems, Nomad -can interact with more unique environments: AWS IAM, dynamic SQL user creation, -etc. all using the same read/write interface. - -## Next - -You now know about secret backends and how to operate on the mount table. -This is important knowledge to move forward and learn about other secret -backends. - -Next, we'll use the -[AWS backend to generate dynamic secrets](/intro/getting-started/dynamic-secrets.html). diff --git a/website/source/layouts/docs.erb b/website/source/layouts/docs.erb index 65c17e8cb..378e5b9a4 100644 --- a/website/source/layouts/docs.erb +++ b/website/source/layouts/docs.erb @@ -38,30 +38,6 @@ > Basic Concepts - > + > Getting Started