kemko/nomad
1
0
Fork 0
mirror of https://github.com/kemko/nomad.git synced 2026-01-06 18:35:44 +03:00
Code Issues Packages Projects Releases Wiki Activity

[17449] Introduces a locking mechanism over variables (#18207)

Browse Source 
It includes the work over the state store, the PRC server, the HTTP server, the go API package and the CLI's  command. To read more on the actuall functionality, refer to the RFCs [NMD-178] Locking with Nomad Variables and [NMD-179] Leader election using locking mechanism for the Autoscaler.
This commit is contained in:
Juana De La Cuesta
2023-09-21 17:56:33 +02:00
committed by GitHub
parent 86d2cdcf80
commit 72acaf6623
40 changed files with 5960 additions and 525 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
website/content/api-docs/variables/index.mdx Normal file
View File

@@ -0,0 +1,21 @@
---
layout: api
page_title: Variables - HTTP API
description: |-
The /var endpoints are used to query for and interact with variables and variable
locking.
---
# Vars HTTP API
The `/var` and `/vars` endpoints are used to query for and interact with
variables, and set up locks and leases over them.
See the [Variables][] documentation for information how these capabilities are
used. For a CLI to perform these operations manually, please see the
documentation for the [`nomad var`][] commands.
Please choose a sub-section in the navigation for more information
[`nomad var`]: /nomad/docs/commands/var
[Variables]: /nomad/docs/concepts/variables

211
website/content/api-docs/variables/locks.mdx Normal file
View File

@@ -0,0 +1,211 @@
---
layout: api
page_title: Variable Locks - HTTP API
description: The /var endpoints are used to query for and interact with variables and locks.
---
# Locks HTTP API
The `/var` endpoint is used to hold, renew and release a lock over a variable.
## Lock Variable
The endpoint to create a variable can also be used to hold a lock and interact with
it through the use of a parameter defining the operation to be performed.
| Method | Path | Produces |
|--------|--------------------------------------|--------------------|
| `PUT` | `/v1/var/:var_path?<lock-operation>` | `application/json` |
### Parameters
The lock operation parameter can be:
- `lock-acquire`: When used, the call will introduce a lock over the variable if
it exists, or create a new one if it doesn't. The lock ID will be returned in the
response and it must be provided to perform any other operation over the lock.
The variable items can be updated at any time using the lock ID, but the lock
parameters are unmmutable, attempting to modify them while a lock is present will
generate an error.
In the case of attempting to acquire a variable that is already locked, a conflict
response will be returned.
The lock-acquire operation will override the variable items if new values are
present.
#### Sample Request
```shell-session
$ curl \
-XPUT -d@spec.nsv.json \
https://localhost:4646/v1/var/example/first?lock-acquire
```
#### Sample Payload
```json
{
"Namespace": "prod",
"Path": "example/first",
"Items": {
"user": "me",
"password": "passw0rd1"
},
"Lock": {
"TTL": "15s",
"LockDelay": "1m"
}
}
```
#### Sample Response
The response body returns the created or updated variable including the lock
parameters and ID, along with metadata created by the server:
```json
{
"CreateIndex": 15,
"CreateTime": 1694552155379696000,
"Items": {
"user": "me",
"password": "passw0rd1"
},
"Lock": {
"TTL": "15s",
"LockDelay": "15s",
"ID": "670c7248-e2ef-f982-e4c5-f4437f75f1e4"
},
"ModifyIndex": 16,
"ModifyTime": 1694552206138804000,
"Namespace": "prod",
"Path": "example/first"
}
```
- `lock-renew`: A valid call to lock renew needs to be placed before the lock's
TTL is up in order to mantain the variable locked. A valid call must include the
lock ID as part of the request body. If the lock TTL is up without a renewal or
release calls, the variable will remain unlockable for at least the lock delay.
#### Sample Request
```shell-session
$ curl \
-XPUT -d@spec.nsv.json \
https://localhost:4646/v1/var/example/first?lock-renew
```
#### Sample Payload
```json
{
"Path": "example/first",
"Namespace": "prod",
"Lock": {
"ID": "670c7248-e2ef-f982-e4c5-f4437f75f1e4"
}
}
```
#### Sample Response
The response body only returns metadata created by the server and the lock
parameters:
```json
{
"CreateIndex": 11,
"CreateTime": 1694555280887153000,
"Lock": {
"TTL": "15s",
"LockDelay": "15s",
"ID": "670c7248-e2ef-f982-e4c5-f4437f75f1e4"
},
"ModifyIndex": 43,
"ModifyTime": 1694556175092779000,
"Namespace": "prod",
"Path": "example/first"
}
```
- `lock-release`: A call to the endpoint with the `lock-release` operation will
immediately remove the lock over the variable, making it modifiable without
restrictions again.
The lock-release operation will not override the variable items, if the request
body contains any item, it will generate a bad request response.
#### Sample Request
```shell-session
$ curl \
-XPUT -d@spec.nsv.json \
https://localhost:4646/v1/var/example/first?lock-release
```
#### Sample Payload
```json
{
"Path": "example/first",
"Namespace": "prod",
"Lock": {
"ID": "670c7248-e2ef-f982-e4c5-f4437f75f1e4"
}
}
```
#### Sample Response
The response body returns the released variable along with metadata
created by the server:
```json
{
"CreateIndex": 11,
"CreateTime": 1694555280887153000,
"ModifyIndex": 66,
"ModifyTime": 1694556922600469000,
"Namespace": "prod",
"Path": "example/first"
}
```
### Sample Response for Conflict
In the case of an attempt to lock, renew or modify a locked variable
without the correct ID, the API will return HTTP error code
409 and a response body showing the conflicting variable. If the provided ACL
token does not also have `read` permissions to the variable path, the response
will include only metadata and not the `Items` field:
```json
{
"CreateIndex": 0,
"CreateTime": 0,
"Items": null,
"Lock": null,
"ModifyIndex": 0,
"ModifyTime": 0,
"Namespace": "default",
"Path": "example/first"
}
```
## Restrictions
When creating a new variable using the lock-acquire operation, all the known
[restrictions][] regarding the path and size of the content apply, but unlike
regular variables, locked variables can be created with or without any items.
The lock TTL and Delay must be values between 10 seconds and 24 hours.
[Variables]: /nomad/docs/concepts/variables
[restrictions]: /nomad/api-docs/variables/variables#restrictions
[`nomad var`]: /nomad/docs/commands/var
[blocking queries]: /nomad/api-docs#blocking-queries
[required ACLs]: /nomad/api-docs#acls
[RFC3986]: https://www.rfc-editor.org/rfc/rfc3986#section-2

300
website/content/api-docs/variables/variables.mdx Normal file
View File

@@ -0,0 +1,300 @@
---
layout: api
page_title: Variables - HTTP API
description: The /var endpoints are used to query for and interact with variables.
---
# Variables HTTP API
The `/var` and `/vars` endpoints are used to query for and interact with
variables.
See the [Variables][] documentation for information how these capabilities are
used.
## List Variables
This endpoint lists all variables. Note this API returns only variable metadata
without decrypting the variable body.
| Method | Path | Produces |
|--------|------------|--------------------|
| `GET` | `/v1/vars` | `application/json` |
The table below shows this endpoint's support for [blocking queries] and
[required ACLs].
| Blocking Queries | ACL Required |
|------------------|-----------------------------------------------------------------------------------------|
| `YES` | `namespace:* variables:list`<br />The list capability on the namespace and path queried |
### Parameters
- `prefix` `(string: "")` - Specifies a string to filter variables on based on
an index prefix. This is specified as a query string parameter.
- `next_token` `(string: "")` - This endpoint supports paging. The `next_token`
parameter accepts a string which identifies the next expected job. This value
can be obtained from the `X-Nomad-NextToken` header from the previous
response.
- `per_page` `(int: 0)` - Specifies a maximum number of variables to return for
this request. If omitted, the response is not paginated. The value of the
`X-Nomad-NextToken` header of the last response can be used as the
`next_token` of the next request to fetch additional pages.
- `filter` `(string: "")` - Specifies the [expression](/nomad/api-docs#filtering) used
to filter the results. Consider using pagination or a query parameter to
reduce resources used to serve the request.
- `namespace` `(string: "default")` - Specifies the target namespace. Specifying
`*` will return all variables across all the authorized namespaces.
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/vars?namespace=prod&prefix=example
```
### Sample Response
```json
[
{
"Namespace": "prod",
"Path": "example/first",
"CreateIndex": 1457,
"ModifyIndex": 1457,
"CreateTime": 1662061225600373000,
"ModifyTime": 1662061225600373000
},
{
"Namespace": "prod",
"Path": "example/second",
"CreateIndex": 800,
"ModifyIndex": 1000,
"CreateTime": 1662061717905426000,
"ModifyTime": 1662062162982630000
}
]
```
## Read Variable
This endpoint reads a specific variable by path. This API returns the decrypted
variable body.
| Method | Path | Produces |
|--------|--------------------|--------------------|
| `GET` | `/v1/var/:var_path` | `application/json` |
The table below shows this endpoint's support for [blocking queries] and
[required ACLs].
| Blocking Queries | ACL Required |
|------------------|--------------------------------------------------------------------------------------------|
| `YES` | `namespace:* variables:read`<br />The read capability on the variable's namespace and path |
### Parameters
- `namespace` `(string: "default")` - Specifies the variable's namespace.
### Sample Request
```shell-session
$ curl \
https://localhost:4646/v1/var/example/first?namespace=prod
```
### Sample Response
```json
{
"Namespace": "prod",
"Path": "example/first",
"CreateIndex": 1457,
"ModifyIndex": 1457,
"CreateTime": 1662061225600373000,
"ModifyTime": 1662061225600373000
"Items": {
"user": "me",
"password": "passw0rd1"
}
}
```
## Create Variable
This endpoint creates or updates a variable.
| Method | Path | Produces |
|--------|---------------------|--------------------|
| `PUT` | `/v1/var/:var_path` | `application/json` |
The table below shows this endpoint's support for [blocking queries] and
[required ACLs].
| Blocking Queries | ACL Required |
|------------------|---------------------------------------------------------------------------------------------|
| `NO` | `namespace:* variables:write`<br />The read capability on the variable's namespace and path |
### Parameters
- `namespace` `(string: "default")` - Specifies the variable's namespace. If
set, this will override the request body.
- `cas` `(int: <unset>)` - If set, the variable will only be updated if the
`cas` value matches the current variables `ModifyIndex`. If the `cas` value is
0, the variable is only created if it does not already exist. This paradigm
allows check-and-set style updates.
- `lock-operation` `(string: <unset>)` - This endpoint can also be used to create
and hold a lock over a variable, refer to the [locks section][] for more
information.
## Restrictions
Variable paths are restricted to [RFC3986][] URL-safe characters that don't
conflict with the use of the characters `@` and `.` in template blocks. This
includes alphanumeric characters and the special characters `-`, `_`, `~`, and
`/`. Paths may be up to 128 bytes long. The following regex matches the allowed
paths: `^[a-zA-Z0-9-_~/]{1,128}$`
Variable items are restricted to 64KiB in size. This limit is calculated by
taking the sum of the length in bytes of all of the unencrypted keys and values
in the `Items` field.
### Sample Request
```shell-session
$ curl \
-XPUT -d@spec.nsv.json \
https://localhost:4646/v1/var/example/first
```
### Sample Payload
```json
{
"Namespace": "prod",
"Path": "example/first",
"Items": {
"user": "me",
"password": "passw0rd1"
}
}
```
### Sample Response
The response body returns the created or updated variable along with metadata
created by the server:
```json
{
"Namespace": "prod",
"Path": "example/first",
"CreateIndex": 1457,
"ModifyIndex": 1457,
"CreateTime": 1662061225600373000,
"ModifyTime": 1662061225600373000,
"Items": {
"user": "me",
"password": "passw0rd1"
}
}
```
### Sample Response for Conflict
In the case of a compare-and-set conflict, the API will return HTTP error code
409 and a response body showing the conflicting variable. If the provided ACL
token does not also have `read` permissions to the variable path, the response
will include only metadata and not the `Items` field:
```json
{
"Namespace": "prod",
"Path": "example/first",
"CreateIndex": 1457,
"ModifyIndex": 1457,
"CreateTime": 1662061225600373000,
"ModifyTime": 1662061225600373000,
"Items": {
"user": "me",
"password": "passw0rd1"
}
}
```
## Delete Variable
This endpoint deletes a specific variable by path.
| Method | Path | Produces |
|--------|--------------------|--------------------|
| `DELETE` | `/v1/var/:var_path` | `application/json` |
The table below shows this endpoint's support for [blocking queries] and
[required ACLs].
| Blocking Queries | ACL Required |
|------------------|--------------------------------------------------------------------------------------------------|
| `NO` | `namespace:* variables:destroy`<br />The destroy capability on the variable's namespace and path |
### Parameters
- `namespace` `(string: "default")` - Specifies the variable's namespace.
- `cas` `(int: <unset>)` - If set, the variable will only be deleted if the
`cas` value matches the current variables `ModifyIndex`.
### Sample Request
```shell-session
$ curl \
-XDELETE \
https://localhost:4646/v1/var/example/first?namespace=prod
```
### Sample Response
```json
{
"Index": 16
}
```
### Sample Request With CAS
```shell-session
$ curl \
-XDELETE \
https://localhost:4646/v1/var/example/first?namespace=prod&cas=1
```
### Sample Response for Conflict
In the case of a compare-and-set conflict on delete, the API will return HTTP
error code 409 and a response body showing the conflicting variable. If the
provided ACL token does not also have `read` permissions to the variable path,
the response will include only metadata and not the `Items` field:
```json
{
"Namespace": "prod",
"Path": "example/first",
"CreateIndex": 1457,
"ModifyIndex": 1457,
"CreateTime": 1662061225600373000,
"ModifyTime": 1662061225600373000
}
```
[Variables]: /nomad/docs/concepts/variables
[locks section]:/nomad/api-docs/variables/locks
[blocking queries]: /nomad/api-docs#blocking-queries
[required ACLs]: /nomad/api-docs#acls
[RFC3986]: https://www.rfc-editor.org/rfc/rfc3986#section-2