mirror of
https://github.com/kemko/nomad.git
synced 2026-01-06 18:35:44 +03:00
* Move commands from docs to its own root-level directory * temporarily use modified dev-portal branch with nomad ia changes * explicitly clone nomad ia exp branch * retrigger build, fixed dev-portal broken build * architecture, concepts and get started individual pages * fix get started section destinations * reference section * update repo comment in website-build.sh to show branch * docs nav file update capitalization * update capitalization to force deploy * remove nomad-vs-kubernetes dir; move content to what is nomad pg * job section * Nomad operations category, deploy section * operations category, govern section * operations - manage * operations/scale; concepts scheduling fix * networking * monitor * secure section * remote auth-methods folder and move up pages to sso; linkcheck * Fix install2deploy redirects * fix architecture redirects * Job section: Add missing section index pages * Add section index pages so breadcrumbs build correctly * concepts/index fix front matter indentation * move task driver plugin config to new deploy section * Finish adding full URL to tutorials links in nav * change SSO to Authentication in nav and file system * Docs NomadIA: Move tutorials into NomadIA branch (#26132) * Move governance and policy from tutorials to docs * Move tutorials content to job-declare section * run jobs section * stateful workloads * advanced job scheduling * deploy section * manage section * monitor section * secure/acl and secure/authorization * fix example that contains an unseal key in real format * remove images from sso-vault * secure/traffic * secure/workload-identities * vault-acl change unseal key and root token in command output sample * remove lines from sample output * fix front matter * move nomad pack tutorials to tools * search/replace /nomad/tutorials links * update acl overview with content from deleted architecture/acl * fix spelling mistake * linkcheck - fix broken links * fix link to Nomad variables tutorial * fix link to Prometheus tutorial * move who uses Nomad to use cases page; move spec/config shortcuts add dividers * Move Consul out of Integrations; move namespaces to govern * move integrations/vault to secure/vault; delete integrations * move ref arch to docs; rename Deploy Nomad back to Install Nomad * address feedback * linkcheck fixes * Fixed raw_exec redirect * add info from /nomad/tutorials/manage-jobs/jobs * update page content with newer tutorial * link updates for architecture sub-folders * Add redirects for removed section index pages. Fix links. * fix broken links from linkcheck * Revert to use dev-portal main branch instead of nomadIA branch * build workaround: add intro-nav-data.json with single entry * fix content-check error * add intro directory to get around Vercel build error * workound for emtpry directory * remove mdx from /intro/ to fix content-check and git snafu * Add intro index.mdx so Vercel build should work --------- Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
222 lines
8.0 KiB
Plaintext
222 lines
8.0 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: parameterized block in the job specification
|
|
description: |-
|
|
A parameterized job is used to encapsulate a set of work that can be carried
|
|
out on various inputs much like a function definition. When the
|
|
`parameterized` block is added to a job, the job acts as a function to the
|
|
cluster as a whole.
|
|
---
|
|
|
|
# `parameterized` block in the job specification
|
|
|
|
<Placement groups={['job', 'parameterized']} />
|
|
|
|
A parameterized job is used to encapsulate a set of work that can be carried out
|
|
on various inputs much like a function definition. When the `parameterized`
|
|
block is added to a job, the job acts as a function to the cluster as a whole.
|
|
|
|
The `parameterized` block allows job operators to configure a job that carries
|
|
out a particular action, define its resource requirements and configure how
|
|
inputs and configuration are retrieved by the tasks within the job.
|
|
|
|
To invoke a parameterized job, [`nomad job dispatch`][dispatch command] or the equivalent HTTP APIs are
|
|
used. When dispatching against a parameterized job, an opaque payload and
|
|
metadata may be injected into the job. These inputs to the parameterized job act
|
|
like arguments to a function. The job consumes them to change its behavior,
|
|
without exposing the implementation details to the caller.
|
|
|
|
To that end, tasks within the job can add a
|
|
[`dispatch_payload`][dispatch_payload] block that
|
|
defines where on the filesystem this payload gets written to. An example payload
|
|
would be a task's JSON configuration.
|
|
|
|
Further, certain metadata may be marked as required when dispatching a job so it
|
|
can be used to inject configuration directly into a task's arguments using
|
|
[interpolation]. An example of this would be to require a run ID key that
|
|
could be used to lookup the work the job is suppose to do from a management
|
|
service or database.
|
|
|
|
Each time a job is dispatched, a unique job ID is generated. This
|
|
allows a caller to track the status of the job, much like a future or
|
|
promise in some programming languages. The dispatched job cannot be
|
|
updated after dispatching; to update the job definition you need to
|
|
update the parent job.
|
|
|
|
```hcl
|
|
job "docs" {
|
|
parameterized {
|
|
payload = "required"
|
|
meta_required = ["dispatcher_email"]
|
|
meta_optional = ["pager_email"]
|
|
}
|
|
}
|
|
```
|
|
|
|
See the [multiregion] documentation for additional considerations when
|
|
dispatching parameterized jobs.
|
|
|
|
## Requirements
|
|
|
|
- The job's [scheduler type][batch-type] must be `batch` or `sysbatch`.
|
|
|
|
## Parameters
|
|
|
|
- `meta_optional` `(array<string>: nil)` - Specifies the set of metadata keys that
|
|
may be provided when dispatching against the job.
|
|
|
|
- `meta_required` `(array<string>: nil)` - Specifies the set of metadata keys that
|
|
must be provided when dispatching against the job.
|
|
|
|
- `payload` `(string: "optional")` - Specifies the requirement of providing a
|
|
payload when dispatching against the parameterized job. The **maximum size of a
|
|
`payload` is 16 KiB**. The options for this
|
|
field are:
|
|
|
|
- `"optional"` - A payload is optional when dispatching against the job.
|
|
|
|
- `"required"` - A payload must be provided when dispatching against the job.
|
|
|
|
- `"forbidden"` - A payload is forbidden when dispatching against the job.
|
|
|
|
## Examples
|
|
|
|
The following examples show non-runnable example parameterized jobs:
|
|
|
|
### Required inputs
|
|
|
|
This example shows a parameterized job that requires both a payload and
|
|
metadata:
|
|
|
|
```hcl
|
|
job "video-encode" {
|
|
# ...
|
|
|
|
type = "batch"
|
|
|
|
parameterized {
|
|
payload = "required"
|
|
meta_required = ["dispatcher_email"]
|
|
}
|
|
|
|
group "encode" {
|
|
# ...
|
|
|
|
task "ffmpeg" {
|
|
driver = "exec"
|
|
|
|
config {
|
|
command = "ffmpeg-wrapper"
|
|
|
|
# When dispatched, the payload is written to a file that is then read by
|
|
# the created task upon startup
|
|
args = ["-config=${NOMAD_TASK_DIR}/config.json"]
|
|
}
|
|
|
|
dispatch_payload {
|
|
file = "config.json"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Metadata interpolation
|
|
|
|
```hcl
|
|
job "email-blast" {
|
|
# ...
|
|
|
|
type = "batch"
|
|
|
|
parameterized {
|
|
payload = "forbidden"
|
|
meta_required = ["CAMPAIGN_ID"]
|
|
}
|
|
|
|
group "emails" {
|
|
# ...
|
|
|
|
task "emailer" {
|
|
driver = "exec"
|
|
|
|
config {
|
|
command = "emailer"
|
|
|
|
# The campaign ID is interpolated and injected into the task's
|
|
# arguments
|
|
args = ["-campaign=${NOMAD_META_CAMPAIGN_ID}"]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Use periodic with parameterized
|
|
|
|
Nomad uses an internal hierarchy when scheduling a job that is both `parameterized` and [`periodic`][periodic].
|
|
|
|
```plaintext
|
|
parameterized => periodic => batch
|
|
```
|
|
|
|
Nomad processes a periodic with parameterized job in the following order:
|
|
|
|
1. Nomad does not dispatch a periodic job with null parameters in the periodic configuration. This forces the parameterized job to take precedence over the periodic job.
|
|
2. After Nomad dispatches the parameterized job and gives it parameters, Nomad uses the periodic configuration.
|
|
3. Nomad dispatches new jobs according to the periodic configuration that uses thee parameters from the triggering parameterized job.
|
|
|
|
In this example, the periodic job does not trigger any new jobs
|
|
until the operator dispatches the parameterized job at least once. After that, the
|
|
dispatched child periodically triggers more children with the given parameters.
|
|
|
|
```hcl
|
|
periodic {
|
|
crons = [
|
|
"*/40 * * * * * *"
|
|
]
|
|
}
|
|
parameterized {
|
|
payload = "required"
|
|
meta_required = ["dispatcher_email"]
|
|
meta_optional = ["pager_email"]
|
|
}
|
|
```
|
|
|
|
|
|
|
|
There are three columns plus comments in this example output, which is for the preceding periodic, parameterized example job. Scroll to the last column to review the comments.
|
|
|
|
```
|
|
$ nomad job status
|
|
ID Type Submit Date
|
|
sync batch/periodic/parameterized 2024-11-07T10:43:30+01:00 // Original submitted job
|
|
sync/dispatch-1730972650-247c6e97 batch/periodic 2024-11-07T10:44:10+01:00 // First dispatched job with parameters A
|
|
sync/dispatch-1730972650-247c6e97/periodic-1730972680 batch 2024-11-07T10:44:40+01:00 // Cron job with parameters A
|
|
sync/dispatch-1730972650-247c6e97/periodic-1730972860 batch 2024-11-07T10:47:40+01:00 // Cron job with parameters A
|
|
sync/dispatch-1730972760-f79a96e1 batch/periodic 2024-11-07T10:46:00+01:00 // Second dispatched job with parameters B
|
|
sync/dispatch-1730972760-f79a96e1/periodic-1730972800 batch 2024-11-07T10:46:40+01:00 // Cron job with parameters B
|
|
sync/dispatch-1730972760-f79a96e1/periodic-1730972860 batch 2024-11-07T10:47:40+01:00 // Cron job with parameters B
|
|
```
|
|
The output illustrates the following:
|
|
|
|
1. The `sync` job is the parameterized and periodic job. However, Nomad doesn't run the periodic job until you submit with the required parameters.
|
|
2. The `sync/dispatch-1730972650-247c6e97` job contains parameters and triggers the batch jobs, named `sync/dispatch-1730972650-247c6e97/periodic-1730972680` and `sync/dispatch-1730972650-247c6e97/periodic-1730972860`.
|
|
3. The `sync/dispatch-1730972760-f79a96e1` job assigns new parameters and triggers new batch jobs that use those parameters.
|
|
|
|
If you need to force the periodic job, force the the corresponding parameterized job.
|
|
|
|
This example forces the first dispatched job with parameters A from the preceding example.
|
|
|
|
```
|
|
$ nomad job periodic force sync/dispatch-1730972650-247c6e97
|
|
```
|
|
|
|
[batch-type]: /nomad/docs/job-specification/job#type 'Batch scheduler type'
|
|
[dispatch command]: /nomad/commands/job/dispatch 'Nomad Job Dispatch Command'
|
|
[resources]: /nomad/docs/job-specification/resources 'Nomad resources Job Specification'
|
|
[interpolation]: /nomad/docs/reference/runtime-variable-interpolation 'Nomad Runtime Interpolation'
|
|
[dispatch_payload]: /nomad/docs/job-specification/dispatch_payload 'Nomad dispatch_payload Job Specification'
|
|
[multiregion]: /nomad/docs/job-specification/multiregion#parameterized-dispatch
|
|
[periodic]: /nomad/docs/job-specification/periodic
|