mirror of
https://github.com/kemko/nomad.git
synced 2026-01-06 18:35:44 +03:00
* Move commands from docs to its own root-level directory * temporarily use modified dev-portal branch with nomad ia changes * explicitly clone nomad ia exp branch * retrigger build, fixed dev-portal broken build * architecture, concepts and get started individual pages * fix get started section destinations * reference section * update repo comment in website-build.sh to show branch * docs nav file update capitalization * update capitalization to force deploy * remove nomad-vs-kubernetes dir; move content to what is nomad pg * job section * Nomad operations category, deploy section * operations category, govern section * operations - manage * operations/scale; concepts scheduling fix * networking * monitor * secure section * remote auth-methods folder and move up pages to sso; linkcheck * Fix install2deploy redirects * fix architecture redirects * Job section: Add missing section index pages * Add section index pages so breadcrumbs build correctly * concepts/index fix front matter indentation * move task driver plugin config to new deploy section * Finish adding full URL to tutorials links in nav * change SSO to Authentication in nav and file system * Docs NomadIA: Move tutorials into NomadIA branch (#26132) * Move governance and policy from tutorials to docs * Move tutorials content to job-declare section * run jobs section * stateful workloads * advanced job scheduling * deploy section * manage section * monitor section * secure/acl and secure/authorization * fix example that contains an unseal key in real format * remove images from sso-vault * secure/traffic * secure/workload-identities * vault-acl change unseal key and root token in command output sample * remove lines from sample output * fix front matter * move nomad pack tutorials to tools * search/replace /nomad/tutorials links * update acl overview with content from deleted architecture/acl * fix spelling mistake * linkcheck - fix broken links * fix link to Nomad variables tutorial * fix link to Prometheus tutorial * move who uses Nomad to use cases page; move spec/config shortcuts add dividers * Move Consul out of Integrations; move namespaces to govern * move integrations/vault to secure/vault; delete integrations * move ref arch to docs; rename Deploy Nomad back to Install Nomad * address feedback * linkcheck fixes * Fixed raw_exec redirect * add info from /nomad/tutorials/manage-jobs/jobs * update page content with newer tutorial * link updates for architecture sub-folders * Add redirects for removed section index pages. Fix links. * fix broken links from linkcheck * Revert to use dev-portal main branch instead of nomadIA branch * build workaround: add intro-nav-data.json with single entry * fix content-check error * add intro directory to get around Vercel build error * workound for emtpry directory * remove mdx from /intro/ to fix content-check and git snafu * Add intro index.mdx so Vercel build should work --------- Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
289 lines
9.5 KiB
Plaintext
289 lines
9.5 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: artifact block in the job specification
|
|
description: |-
|
|
Configure fetching a remote resource in the `artifact` block of the Nomad job specification. Set the artifact destination, the source URL, HTTP headers, fetch options, mode, and whether Nomad should recursively chown the downloaded artifact. Review examples for downloading a file, fetching from a Git repository, unarchiving, verifying checksums, and downloading from an AWS S3-compatible bucket.
|
|
---
|
|
|
|
# `artifact` block in the job specification
|
|
|
|
<Placement groups={['job', 'group', 'task', 'artifact']} />
|
|
|
|
The `artifact` block instructs Nomad to fetch and unpack a remote resource,
|
|
such as a file, tarball, or binary. Nomad downloads artifacts using the popular
|
|
[`go-getter`][go-getter] library, which permits downloading artifacts from a
|
|
variety of locations using a URL as the input source.
|
|
|
|
```hcl
|
|
job "docs" {
|
|
group "example" {
|
|
task "server" {
|
|
artifact {
|
|
source = "https://example.com/file.tar.gz"
|
|
destination = "local/some-directory"
|
|
options {
|
|
checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Nomad supports downloading `http`, `https`, `git`, `hg` and `S3` artifacts. If
|
|
these artifacts are archived (`zip`, `tgz`, `bz2`, `xz`), they are
|
|
automatically unarchived before the starting the task.
|
|
|
|
## Parameters
|
|
|
|
- `destination` `(string: "local/")` - Specifies the directory path to
|
|
download the artifact, relative to the root of the [task's working
|
|
directory]. If omitted, the default value is to place the artifact in
|
|
`local/`. The destination is treated as a directory unless `mode` is set to
|
|
`file`. Source files will be downloaded into that directory path. For more
|
|
details on how the `destination` interacts with task drivers, see the
|
|
[Filesystem internals] documentation.
|
|
|
|
- `mode` `(string: "any")` - One of `any`, `file`, or `dir`. If set to `file`
|
|
the `destination` must be a file, not a directory. By default the
|
|
`destination` will be `local/<filename>`.
|
|
|
|
- `options` `(map<string|string>: nil)` - Specifies configuration parameters to
|
|
fetch the artifact. The key-value pairs map directly to parameters appended to
|
|
the supplied `source` URL. Please see the [`go-getter`
|
|
documentation][go-getter] for a complete list of options and examples.
|
|
|
|
- `headers` `(map<string|string>: nil)` - Specifies HTTP headers to set when
|
|
fetching the artifact using `http` or `https` protocol. Please see the
|
|
[`go-getter` headers documentation][go-getter-headers] for more information.
|
|
|
|
- `source` `(string: <required>)` - Specifies the URL of the artifact to download.
|
|
See [`go-getter`][go-getter] for details.
|
|
|
|
- `chown` `(bool: false)` - Specifies whether Nomad should recursively `chown`
|
|
the downloaded artifact to be owned by the [`task.user`][task_user] uid and
|
|
gid.
|
|
|
|
## Environment
|
|
|
|
The `artifact` downloader by default does not have access to the environment
|
|
variables set for the Nomad client. Manage inheritance of environment variables
|
|
with the [`artifact.set_environment_variables`][client_artifact] client
|
|
configuration.
|
|
|
|
## Operation limits
|
|
|
|
The client [`artifact`][client_artifact] configuration can set limits to
|
|
specific artifact operations to prevent excessive data download or operation
|
|
time.
|
|
|
|
If a task's `artifact` retrieval exceeds one of those limits, the task will be
|
|
interrupted and fail to start. Refer to the task events for more information.
|
|
|
|
## Examples
|
|
|
|
The following examples only show the `artifact` blocks. Remember that the
|
|
`artifact` block is only valid in the placements listed above.
|
|
|
|
### Download file
|
|
|
|
This example downloads the artifact from the provided URL and places it in
|
|
`local/file.txt`. The `local/` path is relative to the [task's working
|
|
directory].
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.txt"
|
|
}
|
|
```
|
|
|
|
To set HTTP headers in the request for the source the optional `headers` field
|
|
can be configured.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.txt"
|
|
|
|
headers {
|
|
User-Agent = "nomad-[${NOMAD_JOB_ID}]-[${NOMAD_GROUP_NAME}]-[${NOMAD_TASK_NAME}]"
|
|
X-Nomad-Alloc = "${NOMAD_ALLOC_ID}"
|
|
}
|
|
}
|
|
```
|
|
|
|
To use HTTP basic authentication, preprend the username and password to the
|
|
hostname in the URL. All special characters, including the username and
|
|
password, must be URL encoded. For example, for a username `exampleUser` and
|
|
the password `pass/word!`:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://exampleUser:pass%2Fword%21@example.com/file.txt"
|
|
}
|
|
```
|
|
|
|
### Download using git
|
|
|
|
This example downloads the artifact from the provided GitHub URL and places it at
|
|
`local/repo`, as specified by the optional `destination` parameter.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "git::https://github.com/hashicorp/nomad-guides"
|
|
destination = "local/repo"
|
|
}
|
|
```
|
|
|
|
To download from a private repo, sshkey needs to be set. The key must be
|
|
base64-encoded string. On Linux, you can run `base64 -w0 <file>` to encode the
|
|
file. Or use [HCL2](/nomad/docs/reference/hcl2)
|
|
expressions to read and encode the key from a file on your machine:
|
|
|
|
```hcl
|
|
artifact {
|
|
# The git:: prefix forces go-getter's protocol detection to use the git ssh
|
|
# protocol. It can also automatically detect the protocol from the domain of
|
|
# some git hosting providers (such as GitHub) without the prefix.
|
|
source = "git::git@bitbucket.org:example/nomad-examples"
|
|
destination = "local/repo"
|
|
options {
|
|
# Make sure that the system known hosts file is populated:
|
|
# ssh-keyscan github.com | sudo tee -a /etc/ssh/ssh_known_hosts
|
|
# https://github.com/hashicorp/go-getter/issues/55
|
|
sshkey = "${base64encode(file("/path/to/private-key"))}"
|
|
}
|
|
}
|
|
```
|
|
|
|
To clone specific refs or at a specific depth, use the `ref` and `depth`
|
|
options:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "git::https://github.com/hashicorp/nomad-guides"
|
|
destination = "local/repo"
|
|
options {
|
|
ref = "main"
|
|
depth = 1
|
|
}
|
|
}
|
|
```
|
|
|
|
### Download and unarchive
|
|
|
|
This example downloads and unarchives the result in `local/file`. Because the
|
|
source URL is an archive extension, Nomad will automatically decompress it:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.tar.gz"
|
|
}
|
|
```
|
|
|
|
To disable automatic unarchiving, set the `archive` option to false:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.tar.gz"
|
|
options {
|
|
archive = false
|
|
}
|
|
}
|
|
```
|
|
|
|
### Download and verify checksums
|
|
|
|
This example downloads an artifact and verifies the resulting artifact's
|
|
checksum before proceeding. If the checksum is invalid, an error will be
|
|
returned.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.zip"
|
|
|
|
options {
|
|
checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Download from an S3-compatible bucket
|
|
|
|
These examples download artifacts from Amazon S3. There are several different
|
|
types of [S3 bucket addressing][s3-bucket-addr] and [S3 region-specific
|
|
endpoints][s3-region-endpoints]. Non-Amazon S3-compatible endpoints like [Minio]
|
|
are supported, but you must explicitly set the "s3::" prefix.
|
|
|
|
This example uses path-based notation on a publicly-accessible bucket:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "s3://my-bucket-example.s3-us-west-2.amazonaws.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
If a bucket requires authentication, you can avoid the use of credentials by
|
|
using [EC2 IAM instance profiles][iam-instance-profiles]. If this is not possible,
|
|
credentials may be supplied via the `options` parameter:
|
|
|
|
```hcl
|
|
artifact {
|
|
options {
|
|
aws_access_key_id = "<id>"
|
|
aws_access_key_secret = "<secret>"
|
|
aws_access_token = "<token>"
|
|
}
|
|
}
|
|
```
|
|
|
|
To force the S3-specific syntax, use the `s3::` prefix:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "s3::https://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
Alternatively you can use virtual hosted style:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "s3://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
[DigitalOcean Spaces][do_spaces] provide S3-compatible object storage and can be
|
|
used in conjunction with the `artifact` block. Given a bucket named
|
|
`my-bucket-example` located in the `sfo3` region and a file named `my_app.tar.gz`,
|
|
the following artifact block will work, provided the bucket and file are public.
|
|
This uses the origin endpoint as detailed directly from Digital Ocean.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://my-bucket-example.sfo3.digitaloceanspaces.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
If the bucket or file are private, the artifact source string needs to be
|
|
modified to add the `s3::` prefix. The bucket name is also moved to form part
|
|
of the URL path component.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "s3::https://sfo3.digitaloceanspaces.com/my-bucket-example/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
|
|
[client_artifact]: /nomad/docs/configuration/client#artifact-parameters
|
|
[go-getter]: https://github.com/hashicorp/go-getter 'HashiCorp go-getter Library'
|
|
[go-getter-headers]: https://github.com/hashicorp/go-getter#headers 'HashiCorp go-getter Headers'
|
|
[minio]: https://www.minio.io/
|
|
[s3-bucket-addr]: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro 'Amazon S3 Bucket Addressing'
|
|
[s3-region-endpoints]: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region 'Amazon S3 Region Endpoints'
|
|
[iam-instance-profiles]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html 'EC2 IAM instance profiles'
|
|
[task's working directory]: /nomad/docs/reference/runtime-environment-settings#task-directories 'Task Directories'
|
|
[task_user]: /nomad/docs/job-specification/task#user
|
|
[filesystem internals]: /nomad/docs/concepts/filesystem#templates-artifacts-and-dispatch-payloads
|
|
[do_spaces]: https://www.digitalocean.com/products/spaces
|