mirror of
https://github.com/kemko/nomad.git
synced 2026-01-08 11:25:41 +03:00
e45a1246cc
- aws secret key is named incorrectly in the target docs.
It needs to match what is in the nomad-autoscaler repo
(see link below), otherwise the autoscaler will default to AWS sdk
behavior, which could end up using an IAM instance profile
or other environment variables instead of what is passed into the
autoscaler config file.
Ref: e60fb5268d/plugins/builtin/target/aws-asg/plugin/plugin.go (L27)
363 lines
12 KiB
Plaintext
363 lines
12 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Target
|
|
sidebar_title: Target
|
|
description: Target plugins determine where the resource to be autoscaled is located.
|
|
---
|
|
|
|
# Target Plugins
|
|
|
|
Target Plugins determine where the resource to be autoscaled is located. All
|
|
target plugins support the `dry-run` policy config parameter which allows a policy
|
|
to be evaluated, but will noop any suggested changes.
|
|
|
|
Below is a list of plugins you can use with the Nomad Autoscaler:
|
|
|
|
- [Nomad Task Group][nomad_task_group_target]
|
|
- [Dynamic Application Sizing Nomad Task][das_task_target]
|
|
- [AWS AutoScaling Group][aws_asg_target]
|
|
- [Azure Virtual Machine Scale Set][azure_vmss_target]
|
|
|
|
## Nomad Task Group Target
|
|
|
|
The Nomad task group target indicates the scalable resource is a Nomad job
|
|
running on a Nomad cluster.
|
|
|
|
### Agent Configuration Options
|
|
|
|
The Nomad target is automatically launched by the Nomad Autoscaler and so the
|
|
following setup is optional.
|
|
|
|
```hcl
|
|
target "nomad" {
|
|
driver = "nomad"
|
|
}
|
|
```
|
|
|
|
### Policy Configuration Options
|
|
|
|
If using the [Nomad job specification scaling stanza][nomad_scaling_stanza] to
|
|
configure the scaling policy, the following section can be omitted as Nomad will
|
|
populate them on job submission.
|
|
|
|
```hcl
|
|
check "example-check" {
|
|
...
|
|
target "nomad" {
|
|
Namespace = "default"
|
|
Job = "example"
|
|
Group = "cache"
|
|
}
|
|
...
|
|
```
|
|
|
|
- `Namespace` `(string: "")` - The namespace in which the job resides as defined
|
|
by the [`namespace` parameter][nomad_namespace_parameter] within the job
|
|
specification.
|
|
|
|
- `Job` `(string: "")` - The job identifier which contains the task group to
|
|
scale as defined within the job specification [`job` stanza][nomad_job_stanza].
|
|
|
|
- `Group` `(string: "")` - The name of the task group to scale as defined in the
|
|
job specification [`group` stanza][nomad_group_stanza].
|
|
|
|
## Dynamic Application Sizing Nomad Task Target
|
|
|
|
<EnterpriseAlert>
|
|
This functionality only exists in Nomad Autoscaler
|
|
Enterprise. This is not present in the open source version of Nomad Autoscaler.
|
|
</EnterpriseAlert>
|
|
|
|
The `app-sizing-nomad` target plugin reports on the current task resource value
|
|
as well as submits recommendations to Nomad via the [recommendations API endpoint][nomad_recommendations_api]
|
|
based on the result of the Dynamic Application Sizing strategy calculations.
|
|
|
|
### Agent Configuration Options
|
|
|
|
The `app-sizing-nomad` plugin is automatically launched by Nomad Autoscaler
|
|
Enterprise and so the following setup is optional.
|
|
|
|
```hcl
|
|
target "app-sizing-nomad" {
|
|
driver = "app-sizing-nomad"
|
|
}
|
|
```
|
|
|
|
### Policy Configuration Options
|
|
|
|
If using the [Nomad job specification scaling stanza][nomad_scaling_stanza] to
|
|
configure the scaling policy for a task resource, the following section can be
|
|
omitted as Nomad will populate them on job submission. Please also refer to the
|
|
[Nomad Autoscaler policy documentation][nomad_autoscaler_policy] for additional
|
|
detail and guidance.
|
|
|
|
```hcl
|
|
check "mem" {
|
|
...
|
|
target "app-sizing-nomad" {
|
|
Namespace = "default"
|
|
Job = "example"
|
|
Group = "cache"
|
|
Task = "Redis"
|
|
Resource = "MemoryMB"
|
|
}
|
|
...
|
|
```
|
|
|
|
- `Region` `(string: "")` - The region in which the job resides as defined
|
|
by the [`region` parameter][nomad_region_parameter] within the job specification.
|
|
|
|
- `Namespace` `(string: "")` - The namespace in which the job resides as defined
|
|
by the [`namespace` parameter][nomad_namespace_parameter] within the job
|
|
specification.
|
|
|
|
- `Job` `(string: "")` - The job identifier which contains the task group to
|
|
scale as defined within the job specification [`job` stanza][nomad_job_stanza].
|
|
|
|
- `Group` `(string: "")` - The name of the task group to scale as defined in the
|
|
job specification [`group` stanza][nomad_group_stanza].
|
|
|
|
- `Task` `(string: "")` - The name of the task to scale as defined in the job
|
|
specification [`task` stanza][nomad_task_stanza].
|
|
|
|
- `Resource` `(string: "")` - The resource to scale as defined by the job
|
|
specification [`resource` stanza][nomad_resource_stanza]. Supports either `CPU`
|
|
or `MemoryMB`.
|
|
|
|
## AWS AutoScaling Group Target
|
|
|
|
The AWS ASG target plugin allows for the scaling of the Nomad cluster clients
|
|
via manipulating [AWS AutoScaling Groups][aws_autoscaling].
|
|
|
|
### Agent Configuration Options
|
|
|
|
To use the AWS ASG target plugin, the agent configuration needs to be populated
|
|
with the appropriate target block. Authentication to the AWS API can be supplied
|
|
in a number of ways including EC2 instance roles. It is recommended, if possible
|
|
to use the [Vault AWS Secrets engine][vault_aws_backend] for supplying access
|
|
credentials to the plugin. Credentials should be injected into the configuration
|
|
via a template rather than as environment variables. This ensures the credentials
|
|
are passed only to the plugin, rather than being available for all plugins and
|
|
the agent process.
|
|
|
|
The IAM policy required for the AWS ASG plugin to function properly is detailed
|
|
below.
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Sid": "",
|
|
"Effect": "Allow",
|
|
"Action": [
|
|
"ec2:TerminateInstances",
|
|
"ec2:DescribeInstanceStatus",
|
|
"autoscaling:UpdateAutoScalingGroup",
|
|
"autoscaling:DetachInstances",
|
|
"autoscaling:DescribeScalingActivities",
|
|
"autoscaling:DescribeAutoScalingGroups",
|
|
"autoscaling:CreateOrUpdateTags"
|
|
],
|
|
"Resource": "*"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
```hcl
|
|
target "aws-asg" {
|
|
driver = "aws-asg"
|
|
config = {
|
|
aws_region = "eu-west-3"
|
|
aws_access_key_id = "AKIAIOSFODNN7EXAMPLE"
|
|
aws_secret_access_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
|
|
}
|
|
}
|
|
```
|
|
|
|
When using a Nomad cluster with ACLs enabled, the plugin will require an ACL token which provides
|
|
the following permissions:
|
|
|
|
```hcl
|
|
node {
|
|
policy = "write"
|
|
}
|
|
```
|
|
|
|
- `aws_region` `(string: "us-east-1")` - The [AWS region][aws_region] identifier
|
|
to connect to and where resources should be managed.
|
|
|
|
- `aws_access_key_id` `(string: "")` - The AWS access key ID used to authenticate
|
|
with the AWS API.
|
|
|
|
- `aws_secret_access_key` `(string: "")` - The AWS secret key ID used to authenticate
|
|
with the AWS API.
|
|
|
|
- `aws_session_token` `(string: "")` - The AWS session token used to authenticate
|
|
with the AWS API.
|
|
|
|
### Policy Configuration Options
|
|
|
|
```hcl
|
|
check "hashistack-allocated-cpu" {
|
|
...
|
|
target "aws-asg" {
|
|
aws_asg_name = "hashistack-client-asg"
|
|
node_class = "hashistack"
|
|
node_drain_deadline = "5m"
|
|
node_purge = "true"
|
|
}
|
|
...
|
|
```
|
|
|
|
- `aws_asg_name` `(string: <required>)` - The name of the AWS AutoScaling Group to
|
|
interact with when performing scaling actions.
|
|
|
|
- `node_class` `(string: <required>)` - The Nomad [client node class][nomad_node_class]
|
|
identifier used to group nodes into a pool of resource.
|
|
|
|
- `node_drain_deadline` `(duration: "15m")` The Nomad [drain deadline][nomad_node_drain_deadline]
|
|
to use when performing node draining actions.
|
|
|
|
- `node_purge` `(bool: "false")` A boolean flag to determine whether Nomad clients should
|
|
be [purged][nomad_node_purge] when performing scale in actions.
|
|
|
|
## Azure Virtual Machine Scale Set Target
|
|
|
|
The Azure VMSS target plugin allows for the scaling of the Nomad cluster clients
|
|
via manipulating [Azure Virtual Machine Scale Sets][azure_virtual_machine_scale_sets].
|
|
|
|
~> The Azure VMSS target plugin requires Azure fingerprinting, which became
|
|
available in Nomad 1.0. If you are running a previous version of Nomad you will
|
|
to set a [meta tag][client_meta_tag] in all of your nodes.
|
|
|
|
### Agent Configuration Options
|
|
|
|
To use the Azure VMSS target plugin, the agent configuration needs to be populated
|
|
with the appropriate target block. Authentication to the Azure API can be supplied
|
|
in a number of ways.
|
|
|
|
#### Virtual Machine Identities
|
|
|
|
When using [virtual machine identities][vm_identity] you will need to provide a
|
|
user-managed identity with the `Contributor` role and set the `subscription_id`
|
|
in the Autoscaler configuration file:
|
|
|
|
```hcl
|
|
target "azure-vmss" {
|
|
driver = "azure-vmss"
|
|
config = {
|
|
subscription_id = "ee0886ab-5cc2-4583-a3f0-c4bfd044ee82"
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Configuration file
|
|
|
|
Credentials should be injected into the configuration via a template rather
|
|
than as environment variables. This ensures the credentials are passed only to
|
|
the plugin, rather than being available for all plugins and the agent process.
|
|
It is recommended, if possible to use the
|
|
[Vault Azure Secrets engine][vault_azure_backend] for supplying access
|
|
credentials to the plugin.
|
|
|
|
```hcl
|
|
target "azure-vmss" {
|
|
driver = "azure-vmss"
|
|
config = {
|
|
tenant_id = "02e99e69-91b0-4d0a-bd61-bf5c08fbfbab"
|
|
client_id = "34c276fa-2a74-4cf9-9fce-4e60642274cb"
|
|
secret_access_key = "ThisIrznTIS_~FAKEj4X93FTyAgilUC511"
|
|
subscription_id = "ee0886ab-5cc2-4583-a3f0-c4bfd044ee82"
|
|
}
|
|
}
|
|
```
|
|
|
|
- `tenant_id` `(string: "")` - The ID of the tenant to authenticate to.
|
|
|
|
- `client_id` `(string: "")` - The app ID of the user-assigned identity in Azure
|
|
Active Directory.
|
|
|
|
- `secret_access_key` `(string: "")` - The secret key ID used to authenticate
|
|
with the Azure API.
|
|
|
|
- `subscription_id` `(string: "")` - The ID of the subscription to authenticate
|
|
to.
|
|
|
|
#### Nomad ACL
|
|
|
|
When using a Nomad cluster with ACLs enabled, the plugin will require an ACL token
|
|
which provides the following permissions:
|
|
|
|
```hcl
|
|
node {
|
|
policy = "write"
|
|
}
|
|
```
|
|
|
|
### Policy Configuration Options
|
|
|
|
```hcl
|
|
check "clients-azure-vmss" {
|
|
...
|
|
target "azure-vmss" {
|
|
resource_group = "prod"
|
|
vm_scale_set = "hashistack-client-set"
|
|
node_class = "hashistack"
|
|
node_drain_deadline = "5m"
|
|
node_purge = "true"
|
|
}
|
|
...
|
|
```
|
|
|
|
- `resource_group` `(string: <required>)` - The name of the [Azure resource group][azure_resource_group]
|
|
within which the virtual machine scale set resides.
|
|
|
|
- `vm_scale_set` `(string: <required>)` - The name of the Azure virtual machine
|
|
scale set to interact with when performing scaling actions.
|
|
|
|
- `node_class` `(string: <required>)` - The Nomad [client node class][nomad_node_class]
|
|
identifier used to group nodes into a pool of resource.
|
|
|
|
- `node_drain_deadline` `(duration: "15m")` The Nomad [drain deadline][nomad_node_drain_deadline]
|
|
to use when performing node draining actions.
|
|
|
|
- `node_purge` `(bool: "false")` A boolean flag to determine whether Nomad clients should
|
|
be [purged][nomad_node_purge] when performing scale in actions.
|
|
|
|
### Client meta tag
|
|
|
|
The Azure VMSS target plugin requires Nomad 1.0 for Azure fingerprinting.
|
|
Alternatively, you can set a [client meta tag][client_meta_tag_docs] called
|
|
`unique.platform.azure.name` with the value being the Azure VMSS node name.
|
|
This can be retrieved programatically via the
|
|
[Azure Instance Metadata service][azure_instance_metadata].
|
|
|
|
[nomad_node_class]: /docs/configuration/client#node_class
|
|
[nomad_node_drain_deadline]: /api-docs/nodes#deadline
|
|
[nomad_scaling_stanza]: /docs/job-specification/scaling
|
|
[nomad_group_stanza]: /docs/job-specification/group#group-stanza
|
|
[nomad_job_stanza]: /docs/job-specification/job#job-stanza
|
|
[aws_region]: https://aws.amazon.com/about-aws/global-infrastructure/regions_az/
|
|
[aws_autoscaling]: https://aws.amazon.com/autoscaling/
|
|
[vault_aws_backend]: https://www.vaultproject.io/docs/secrets/aws
|
|
[nomad_node_purge]: /api-docs/nodes#purge-node
|
|
[nomad_recommendations_api]: /api-docs/recommendations
|
|
[nomad_task_stanza]: /docs/job-specification/task#task-stanza
|
|
[nomad_resource_stanza]: /docs/job-specification/resources#resources-stanza
|
|
[nomad_namespace_parameter]: /docs/job-specification/job#namespace
|
|
[nomad_region_parameter]: /docs/job-specification/job#region
|
|
[nomad_autoscaler_policy]: /docs/autoscaling/policy
|
|
[azure_virtual_machine_scale_sets]: https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/overview
|
|
[vault_azure_backend]: https://www.vaultproject.io/docs/secrets/azure
|
|
[azure_resource_group]: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal#what-is-a-resource-group
|
|
[nomad_task_group_target]: /docs/autoscaling/plugins/target#nomad-task-group-target
|
|
[aws_asg_target]: /docs/autoscaling/plugins/target#aws-autoscaling-group-target
|
|
[azure_vmss_target]: /docs/autoscaling/plugins/target#azure-virtual-machine-scale-set-target
|
|
[vm_identity]: https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/
|
|
[client_meta_tag_docs]: https://www.nomadproject.io/docs/configuration/client#meta
|
|
[client_meta_tag]: /docs/autoscaling/plugins/target#client-meta-tag
|
|
[azure_instance_metadata]: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service
|
|
[das_task_target]: /docs/autoscaling/plugins/target#dynamic-application-sizing-nomad-task-target
|