mirror of
https://github.com/kemko/nomad.git
synced 2026-01-04 17:35:43 +03:00
website: documentation for autoscaler plugins
This commit is contained in:
Chris Baker
@@ -11,3 +11,7 @@ description: >
|
||||
|
||||
This section covers the internals of the Nomad Autoscaler and explains the
|
||||
technical details of how it functions, its architecture, and sub-systems.
|
||||
|
||||
- [Autoscaler plugins](/docs/autoscaxling/internals/plugins)
|
||||
- [Check calculations](/docs/autoscaling/interals/checks)
|
||||
|
||||
|
||||
48
website/content/docs/autoscaling/internals/plugins/apm.mdx
Normal file
48
website/content/docs/autoscaling/internals/plugins/apm.mdx
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: APM Plugins
|
||||
sidebar_title: APM
|
||||
description: Learn how to author a Nomad Autoscaler APM plugin.
|
||||
---
|
||||
|
||||
# APM Plugins
|
||||
|
||||
APM plugins are used by the autoscaler to interact with an external APM system,
|
||||
returning metrics that are used by the autoscaler to inform scaling actions.
|
||||
|
||||
For a real-world example of a Nomad APM plugin implementation, see the
|
||||
[Prometheus plugin](https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/builtin/apm/prometheus).
|
||||
|
||||
## Authoring APM Plugins
|
||||
|
||||
Authoring an APM plugin in Go can be accomplished by implementing the
|
||||
[apm.APM][apm_plugin] interface, alongside a main package to launch the plugin.
|
||||
|
||||
The [no-op APM plugin][noop_plugin] can be used as a starting point for new APM
|
||||
plugins.
|
||||
|
||||
## APM Plugin API
|
||||
|
||||
The [base plugin][base_plugin] interface must be implemented in addition to the
|
||||
following functions.
|
||||
|
||||
### `Query(query string, timeRange sdk.TimeRange) (sdk.TimestampedMetrics, error)`
|
||||
|
||||
The `Query` function is called by the agent during policy
|
||||
evaluation. The `query` argument is the opaque string from the scaling policy,
|
||||
and the `timeRange` indicates the period of time over which the query should be
|
||||
made. The response is a series of timestamped metrics; as such, the query semantics
|
||||
should be such that the backing APM will return a time series. An example is the
|
||||
CPU utilization of a task group, averaged over all current allocations.
|
||||
|
||||
### `QueryMultiple(query string, timeRange sdk.TimeRange) ([]sdk.TimestampedMetrics, error)`
|
||||
|
||||
The `QueryMultiple` method is similar to `Query`, except that the interface allows
|
||||
multiple time series to be returned. This endpoint is currently only used by [DAS][das].
|
||||
An example would be to return the CPU utilization for all allocations during
|
||||
the time range.
|
||||
|
||||
[apm_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/apm/apm.go#L11
|
||||
[base_plugin]: /docs/autoscaling/internals/plugins/base
|
||||
[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-apm
|
||||
[das]: /docs/autoscaling#dynamic-application-sizing
|
||||
34
website/content/docs/autoscaling/internals/plugins/base.mdx
Normal file
34
website/content/docs/autoscaling/internals/plugins/base.mdx
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Base Plugin
|
||||
sidebar_title: Base
|
||||
description: Learn about how to author a Nomad Autoscaler plugin.
|
||||
---
|
||||
|
||||
# Base Plugin
|
||||
|
||||
The base plugin is an interface required of all autoscaler plugins,
|
||||
providing a set of common operations.
|
||||
|
||||
## Plugin API
|
||||
|
||||
#### `PluginInfo() (*PluginInfo, error)`
|
||||
|
||||
A `PluginInfo` contains metadata about the plugin. For example,
|
||||
the Prometheus APM plugin returns the following;
|
||||
|
||||
```go
|
||||
PluginInfo{
|
||||
// Name of the plugin
|
||||
Name: "prometheus",
|
||||
// Plugin type: "apm", "strategy", or "target"
|
||||
PluginType: "apm"
|
||||
}
|
||||
```
|
||||
|
||||
#### `SetConfig(config map[string]string) error`
|
||||
|
||||
The `SetConfig` function is called when starting an instance of the plugin. It contains the
|
||||
configuration for a named instance of the plugin as provided in the autoscaler [agent config][plugin_config].
|
||||
|
||||
[plugin_config]: /docs/autoscaling/agent
|
||||
40
website/content/docs/autoscaling/internals/plugins/index.mdx
Normal file
40
website/content/docs/autoscaling/internals/plugins/index.mdx
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Plugins
|
||||
sidebar_title: Plugins
|
||||
description: Learn about how external plugins work in the Nomad Autoscaler.
|
||||
---
|
||||
|
||||
# Plugins
|
||||
|
||||
The Nomad Autoscaler uses a plugin framework which allows users to extend its
|
||||
functionality. The design of the plugin system is inspired by
|
||||
[plugin system][nomad_plugin_system] used in Nomad for task drivers and
|
||||
devices.
|
||||
|
||||
The following components are currently pluggable within Nomad:
|
||||
|
||||
- [APMs](/docs/autoscaling/internals/plugins/apm)
|
||||
- [Strategies](/docs/autoscaling/internals/plugins/strategy)
|
||||
- [Targets](/docs/autoscaling/internals/plugins/target)
|
||||
|
||||
In addition, each plugin implements a [base](/docs/autoscaling/internals/plugins/base) plugin functionality
|
||||
|
||||
# Architecture
|
||||
|
||||
The Nomad Autoscaler plugin framework uses the [go-plugin][goplugin] project to expose
|
||||
a language-independent plugin interface. Plugins implement a set of gRPC
|
||||
services and methods which the Autoscaler agent manages by running the plugin
|
||||
and calling the implemented RPCs. This means that plugins are free to be
|
||||
implemented in the author's language of choice.
|
||||
|
||||
To make plugin development easier, a set of go interfaces and structs exist for
|
||||
each plugin type that abstract the go-plugin and gRPC interfaces. The guides in
|
||||
this documentation reference these abstractions for ease of use.
|
||||
|
||||
The existing plugins can serve as examples; in addition, no-op external plugins
|
||||
are available in the [autoscaler repo][noop_plugins].
|
||||
|
||||
[goplugin]: https://github.com/hashicorp/go-plugin
|
||||
[nomad_plugin_system]: /docs/internals/plugins
|
||||
[noop_plugins]: https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/test
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Strategy Plugins
|
||||
sidebar_title: Strategy
|
||||
description: Learn how to author a Nomad Autoscaler Strategy plugin.
|
||||
---
|
||||
|
||||
# Strategy Plugins
|
||||
|
||||
Strategy plugins are used by the autoscaler to implement the logic of scaling strategy.
|
||||
They typically consume metrics from the configured APM plugin and the current scaling value
|
||||
from the target plugin and compute a new desired value for the scaling target.
|
||||
|
||||
For a real-world example of a Nomad strategy plugin implementation, see the
|
||||
[`target-value` plugin][target_value].
|
||||
|
||||
## Authoring Strategy Plugins
|
||||
|
||||
Authoring a strategy plugin in Go can be accomplished by implementing the
|
||||
[strategy.Strategy][strategy_plugin] interface, alongside a main package to launch the plugin.
|
||||
|
||||
The [no-op Strategy plugin][noop_plugin] can be used as a starting point for new Strategy
|
||||
plugins.
|
||||
|
||||
## Strategy Plugin API
|
||||
|
||||
The [base plugin][base_plugin] interface must be implemented in addition to the
|
||||
following functions.
|
||||
|
||||
### ` Run(eval *sdk.ScalingCheckEvaluation, count int64) (*sdk.ScalingCheckEvaluation, error)`
|
||||
|
||||
The `Run` function is called by the agent during policy
|
||||
evaluation. The `eval` argument contains [information](https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/eval_oss.go#L8)
|
||||
about the current policy evaluation, including the policy specification and the metrics from the APM. The `count`
|
||||
argument includes the current value of the scaling target. The returned struct should include the result
|
||||
from applying the strategy.
|
||||
|
||||
[strategy_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/strategy/strategy.go#L11
|
||||
[base_plugin]: /docs/autoscaling/internals/plugins/base
|
||||
[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-strategy
|
||||
[target_value]: https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/builtin/strategy/target-value
|
||||
@@ -0,0 +1,46 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Target Plugins
|
||||
sidebar_title: Target
|
||||
description: Learn how to author a Nomad Autoscaler Target plugin.
|
||||
---
|
||||
|
||||
# Target Plugins
|
||||
|
||||
Target plugins are used by the autoscaler to perform a scaling action against a
|
||||
scaling target.
|
||||
|
||||
For a real-world example of a Nomad Target plugin implementation, see the
|
||||
[Nomad group->count plugin](https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/builtin/target/nomad).
|
||||
|
||||
## Authoring Target Plugins
|
||||
|
||||
Authoring a target plugin in Go can be accomplished by implementing the
|
||||
[target.Target][target_plugin] interface, alongside a main package to launch the plugin.
|
||||
|
||||
The [no-op Target plugin][noop_plugin] can be used as a starting point for new Target
|
||||
plugins.
|
||||
|
||||
## Target Plugin API
|
||||
|
||||
The [base plugin][base_plugin] interface must be implemented in addition to the
|
||||
following functions.
|
||||
|
||||
### `Scale(action sdk.ScalingAction, config map[string]string) error`
|
||||
|
||||
The `Scale` method is called by the agent during policy
|
||||
evaluation. The `action` argument specifies the details about the scaling action
|
||||
that should be made against the target (e.g., new value, metadata). Config includes
|
||||
details about the scaling target (e.g., Nomad job name, AWS autoscaling group).
|
||||
|
||||
### `Status(config map[string]string) (*sdk.TargetStatus, error)`
|
||||
|
||||
The `Status` method is called by the agent in order to determine the current
|
||||
status of a scaling target. This is performed as part of policy evaluation,
|
||||
and the information returned may be used by the scaling strategy to inform the
|
||||
next scaling action. Information returned includes current scaling level,
|
||||
readiness, and arbitrary metadata.
|
||||
|
||||
[target_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/target/target.go#L12
|
||||
[base_plugin]: /docs/autoscaling/internals/plugins/base
|
||||
[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-target
|
||||
17
website/content/docs/autoscaling/plugins/external/index.mdx
vendored
Normal file
17
website/content/docs/autoscaling/plugins/external/index.mdx
vendored
Normal file
@@ -0,0 +1,17 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: 'Autoscaler Plugins: Community Supported'
|
||||
sidebar_title: Community
|
||||
description: A list of community-supported Autoscaler Plugins.
|
||||
---
|
||||
|
||||
# Community Supported
|
||||
|
||||
If you have authored an autoscaler plugin that you believe will be useful to the
|
||||
broader Nomad community and you are committed to maintaining the plugin, please
|
||||
file a PR to add your plugin to this page.
|
||||
|
||||
For details on authoring an autoscaler plugin, please refer to the [plugin
|
||||
authoring guide][plugin_guide].
|
||||
|
||||
[plugin_guide]: /docs/autoscaling/internals/plugins
|
||||
@@ -9,15 +9,17 @@ description: Plugins are used to architect the Nomad Autoscaler into distinct ar
|
||||
|
||||
Plugins are an essential part of the Nomad Autoscaler architecture. The Autoscaler
|
||||
uses the [go-plugin][go_plugin_github] library to implement an ecosystem of
|
||||
different types of plugins. Each plugin type is responsible for a specific task;
|
||||
APM plugins retrieve metrics about the workloads being monitored and Strategy
|
||||
plugins decide which actions Nomad should execute to keep the policy valid. The
|
||||
flexibility of plugins allows the Nomad Autoscaler to be extended to meet specific
|
||||
business requirements or technology use cases.
|
||||
different types of plugins. Each plugin type is responsible for a specific task:
|
||||
APM plugins retrieve metrics about the workloads being monitored; strategy
|
||||
plugins decide the scaling action to satisfy the scaling policy; and target
|
||||
plugins perform the scaling action. The flexibility of plugins allows the Nomad
|
||||
Autoscaler to be extended to meet specific business requirements or technology
|
||||
use cases.
|
||||
|
||||
The Nomad Autoscaler currently ships with a number of built-in plugins to ease
|
||||
the learning curve. Details of these can be found in the side menu, under the
|
||||
specific plugin type sections.
|
||||
specific plugin type sections. The autoscaler also supports external plugins; see
|
||||
this list of [community-supported plugins][community_plugins].
|
||||
|
||||
# General Options
|
||||
|
||||
@@ -63,3 +65,4 @@ targets.
|
||||
strongly discouraged.
|
||||
|
||||
[go_plugin_github]: https://github.com/hashicorp/go-plugin
|
||||
[community_plugins]: /docs/autoscaling/plugins/external
|
||||
|
||||
@@ -19,6 +19,8 @@ Below is a list of plugins you can use with the Nomad Autoscaler:
|
||||
- [Azure Virtual Machine Scale Set][azure_vmss_target]
|
||||
- [Google Cloud Platform Managed Instance Groups][gcp_mig_target]
|
||||
|
||||
|
||||
|
||||
## Nomad Task Group Target
|
||||
|
||||
The Nomad task group target indicates the scalable resource is a Nomad job
|
||||
|
||||
@@ -419,11 +419,30 @@ export default [
|
||||
'telemetry',
|
||||
{
|
||||
category: 'plugins',
|
||||
content: ['apm', 'strategy', 'target'],
|
||||
content: [
|
||||
'apm',
|
||||
'strategy',
|
||||
'target',
|
||||
{
|
||||
category: 'external',
|
||||
content: [],
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
category: 'internals',
|
||||
content: ['checks'],
|
||||
content: [
|
||||
'checks',
|
||||
{
|
||||
category: 'plugins',
|
||||
content: [
|
||||
'base',
|
||||
'apm',
|
||||
'strategy',
|
||||
'target',
|
||||
],
|
||||
}
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user