From 45cb749cd36b9356c544f95a577878afc0c5759a Mon Sep 17 00:00:00 2001 From: Shantanu Gadgil Date: Mon, 18 May 2020 11:38:52 +0530 Subject: [PATCH 1/5] update docs to mention spread algorithm update docs to mention spread algorithm --- website/pages/docs/configuration/server.mdx | 3 +++ 1 file changed, 3 insertions(+) diff --git a/website/pages/docs/configuration/server.mdx b/website/pages/docs/configuration/server.mdx index 4999355ff..4c1719c0b 100644 --- a/website/pages/docs/configuration/server.mdx +++ b/website/pages/docs/configuration/server.mdx @@ -241,6 +241,9 @@ server { ### Configuring Scheduler Config +- `scheduler_algorithm` `(string: "binpack")` - Specifies the schedulaer algorithm to place + allocations. The default value is `binpack`. Possible values are `binpack` and `spread`. + This example shows enabling preemption for all schedulers. ```hcl From 860fb4d35b42df7a17de138f126bc228807b9b4a Mon Sep 17 00:00:00 2001 From: Shantanu Gadgil Date: Mon, 18 May 2020 17:54:27 +0530 Subject: [PATCH 2/5] Update server.mdx --- website/pages/docs/configuration/server.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/pages/docs/configuration/server.mdx b/website/pages/docs/configuration/server.mdx index 4c1719c0b..ed0b44293 100644 --- a/website/pages/docs/configuration/server.mdx +++ b/website/pages/docs/configuration/server.mdx @@ -241,7 +241,7 @@ server { ### Configuring Scheduler Config -- `scheduler_algorithm` `(string: "binpack")` - Specifies the schedulaer algorithm to place +- `scheduler_algorithm` `(string: "binpack")` - Specifies the scheduler algorithm to place allocations. The default value is `binpack`. Possible values are `binpack` and `spread`. This example shows enabling preemption for all schedulers. From 8623725397e16815f5aaa5d0e6b099488ac0fbaa Mon Sep 17 00:00:00 2001 From: Charlie Voiselle <464492+angrycub@users.noreply.github.com> Date: Mon, 18 May 2020 09:51:16 -0400 Subject: [PATCH 3/5] Alternative treatment A variation of the treatment that surfaces "spread scheduling" in the example while referring to the API docs as canonical for this setting. --- website/pages/docs/configuration/server.mdx | 31 ++++++++++++--------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/website/pages/docs/configuration/server.mdx b/website/pages/docs/configuration/server.mdx index ed0b44293..42f4d031d 100644 --- a/website/pages/docs/configuration/server.mdx +++ b/website/pages/docs/configuration/server.mdx @@ -239,17 +239,30 @@ server { } ``` -### Configuring Scheduler Config +### Bootstrapping with a Custom Scheduler Config -- `scheduler_algorithm` `(string: "binpack")` - Specifies the scheduler algorithm to place - allocations. The default value is `binpack`. Possible values are `binpack` and `spread`. +While bootstrapping a node, you can use the `default_scheduler_config` stanza to +prime the cluster with a [`SchedulerConfig`][update-scheduler-config]. The +scheduler configuration determines which scheduling algorithm is configured— +spread scheduling or binpacking—and which job types are eligible for +preemption. -This example shows enabling preemption for all schedulers. +~> **Warning:** Once the cluster is bootstrapped, you must configure this using + the [update scheduler configuration][update-scheduler-config] API. This + option is only consulted during bootstrap. + +The structure matches the [Update Scheduler Config][update-scheduler-config] +API endpoint, which you should consult for canonical documentation. However, the attributes +names must be adapted to HCL syntax by using snake case representations rather than +camel case. + +This example shows configuring spread scheduling and enabling preemption for all +job-type schedulers. ```hcl server { default_scheduler_config { - scheduler_algorithm = "binpack" + scheduler_algorithm = "spread" preemption_config { batch_scheduler_enabled = true @@ -260,14 +273,6 @@ server { } ``` -The structure matches the [Update Scheduler Config][update-scheduler-config] endpoint, -but adopted to hcl syntax (namely using snake case rather than camel case). - -Nomad servers check their `default_scheduler_config` only during cluster -bootstrap. During upgrades, if a previously bootstrapped cluster already set -scheduler configuration via the [Update Scheduler Config][update-scheduler-config] -endpoint, that is always preferred. - [encryption]: https://learn.hashicorp.com/nomad/transport-security/gossip-encryption 'Nomad Encryption Overview' [server-join]: /docs/configuration/server_join 'Server Join' [update-scheduler-config]: /api-docs/operator#update-scheduler-configuration 'Scheduler Config' From 2e4919d6169fae681acef577243577abc6f5f36a Mon Sep 17 00:00:00 2001 From: Charlie Voiselle <464492+angrycub@users.noreply.github.com> Date: Tue, 19 May 2020 09:14:17 -0400 Subject: [PATCH 4/5] Adding info about bootstrap --- website/pages/docs/configuration/server.mdx | 18 ++++++++-------- website/pages/docs/faq.mdx | 23 +++++++++++++++++++++ 2 files changed, 32 insertions(+), 9 deletions(-) diff --git a/website/pages/docs/configuration/server.mdx b/website/pages/docs/configuration/server.mdx index 42f4d031d..f30d49ef2 100644 --- a/website/pages/docs/configuration/server.mdx +++ b/website/pages/docs/configuration/server.mdx @@ -1,4 +1,4 @@ --- +--- layout: docs page_title: server Stanza - Agent Configuration sidebar_title: server @@ -241,20 +241,19 @@ server { ### Bootstrapping with a Custom Scheduler Config -While bootstrapping a node, you can use the `default_scheduler_config` stanza to -prime the cluster with a [`SchedulerConfig`][update-scheduler-config]. The +While [bootstrapping a cluster], you can use the `default_scheduler_config` stanza +to prime the cluster with a [`SchedulerConfig`][update-scheduler-config]. The scheduler configuration determines which scheduling algorithm is configured— -spread scheduling or binpacking—and which job types are eligible for -preemption. +spread scheduling or binpacking—and which job types are eligible for preemption. ~> **Warning:** Once the cluster is bootstrapped, you must configure this using the [update scheduler configuration][update-scheduler-config] API. This option is only consulted during bootstrap. -The structure matches the [Update Scheduler Config][update-scheduler-config] -API endpoint, which you should consult for canonical documentation. However, the attributes -names must be adapted to HCL syntax by using snake case representations rather than -camel case. +The structure matches the [Update Scheduler Config][update-scheduler-config] API +endpoint, which you should consult for canonical documentation. However, the +attributes names must be adapted to HCL syntax by using snake case +representations rather than camel case. This example shows configuring spread scheduling and enabling preemption for all job-type schedulers. @@ -276,3 +275,4 @@ server { [encryption]: https://learn.hashicorp.com/nomad/transport-security/gossip-encryption 'Nomad Encryption Overview' [server-join]: /docs/configuration/server_join 'Server Join' [update-scheduler-config]: /api-docs/operator#update-scheduler-configuration 'Scheduler Config' +[bootstrapping a cluster]: /docs/faq#bootstrapping \ No newline at end of file diff --git a/website/pages/docs/faq.mdx b/website/pages/docs/faq.mdx index abc413741..f8e13bef2 100644 --- a/website/pages/docs/faq.mdx +++ b/website/pages/docs/faq.mdx @@ -37,6 +37,29 @@ Consul on the other hand does not have this two-tier approach to servers and agents and instead [relies on federation to create larger logical clusters][consul_fed]. +## Q: What is "bootstrapping" a Nomad cluster? ((#bootstrapping)) + +Bootstrapping is the process when a Nomad cluster elects its first leader +and writes the initial cluster state to that leader's state store. Bootstrapping +will not occur until at least a given number of servers, defined by +[`bootstrap_expect`], have connected to each other. Once this process has +completed, the cluster is said to be bootstrapped and is ready to use. + +Certain configuration options are only used to influence the creation of the +initial cluster state during bootstrapping and are not consulted again so long +as the state data remains intact. These typically are values that must be +consistent across server members. For example, the [`default_scheduler_config`] +option allows an operator to set the SchedulerConfig to non-default values +during this bootstrap process rather than requiring an immediate call to the API +once the cluster is up and running. + +If the state is completely destroyed, whether intentionally or accidentally, on +all of the Nomad servers in the same outage, the cluster will re-bootstrap based +on the Nomad defaults and any configuration present that impacts the bootstrap +process. + [consul_dc]: https://www.consul.io/docs/agent/options.html#_datacenter [consul_fed]: https://www.consul.io/docs/guides/datacenters.html [nomad_region]: /docs/configuration#datacenter +[`bootstrap_expect`]: /docs/configuration/server#bootstrap_expect +[`default_scheduler_config`]: /docs/configuration/server#default_scheduler_config From 27c12a6e70f2bae2ea126ca302ba93437a6d1fa9 Mon Sep 17 00:00:00 2001 From: Charlie Voiselle <464492+angrycub@users.noreply.github.com> Date: Tue, 19 May 2020 09:14:17 -0400 Subject: [PATCH 5/5] Interlinking API, FAQ, and Config Creating a FAQ question to provide a home for additional context around bootstrapping. Linking from API page to `default_server_config` attribute. Added sample API response to to discuss "Updated: false" --- website/pages/api-docs/operator.mdx | 49 +++++++++++++++++---- website/pages/docs/configuration/server.mdx | 2 +- 2 files changed, 42 insertions(+), 9 deletions(-) diff --git a/website/pages/api-docs/operator.mdx b/website/pages/api-docs/operator.mdx index 6ae229d68..ff26ff5f3 100644 --- a/website/pages/api-docs/operator.mdx +++ b/website/pages/api-docs/operator.mdx @@ -406,6 +406,12 @@ The table below shows this endpoint's support for | ---------------- | ---------------- | | `NO` | `operator:write` | +### Bootstrap Configuration Element + +The [`default_scheduler_config`][] attribute of the server stanza will provide a +starting value for this configuration. Once bootstrapped, the value in the +server state is authoritative. + ### Parameters - `cas` `(int: 0)` - Specifies to use a Check-And-Set operation. The update will @@ -425,11 +431,38 @@ The table below shows this endpoint's support for } ``` -- `SchedulerAlgorithm` `(string: "binpack")` - Specifies whether scheduler binpacks or spreads allocations on available nodes. Possible values are `"binpack"` and `"spread"` -- `PreemptionConfig` `(PreemptionConfig)` - Options to enable preemption for various schedulers. -- `SystemSchedulerEnabled` `(bool: true)` - Specifies whether preemption for system jobs is enabled. Note that - if this is set to true, then system jobs can preempt any other jobs. -- `BatchSchedulerEnabled` `(bool: false)` (Enterprise Only) - Specifies whether preemption for batch jobs is enabled. Note that - if this is set to true, then batch jobs can preempt any other jobs. -- `ServiceSchedulerEnabled` `(bool: false)` (Enterprise Only) - Specifies whether preemption for service jobs is enabled. Note that - if this is set to true, then service jobs can preempt any other jobs. +- `SchedulerAlgorithm` `(string: "binpack")` - Specifies whether scheduler + binpacks or spreads allocations on available nodes. Possible values are + `"binpack"` and `"spread"` + +- `PreemptionConfig` `(PreemptionConfig)` - Options to enable preemption for + various schedulers. + + - `SystemSchedulerEnabled` `(bool: true)` - Specifies whether preemption for + system jobs is enabled. Note that if this is set to true, then system jobs + can preempt any other jobs. + + - `BatchSchedulerEnabled` `(bool: false)` (Enterprise Only) - Specifies + whether preemption for batch jobs is enabled. Note that if this is set to + true, then batch jobs can preempt any other jobs. + + - `ServiceSchedulerEnabled` `(bool: false)` (Enterprise Only) - Specifies + whether preemption for service jobs is enabled. Note that if this is set to + true, then service jobs can preempt any other jobs. + +### Sample Response + +```json +{ + "Updated": false, + "Index": 15 +} +``` + +- `Updated` - Indicates that the configuration was updated when a `cas` value is + provided. For non-CAS requests, this field will be false even though the + update is applied. + +- `Index` - Current Raft index when the request was received. + +[`default_scheduler_config`]: /docs/configuration/server#default_scheduler_config \ No newline at end of file diff --git a/website/pages/docs/configuration/server.mdx b/website/pages/docs/configuration/server.mdx index f30d49ef2..300910962 100644 --- a/website/pages/docs/configuration/server.mdx +++ b/website/pages/docs/configuration/server.mdx @@ -239,7 +239,7 @@ server { } ``` -### Bootstrapping with a Custom Scheduler Config +### Bootstrapping with a Custom Scheduler Config ((#configuring-scheduler-config)) While [bootstrapping a cluster], you can use the `default_scheduler_config` stanza to prime the cluster with a [`SchedulerConfig`][update-scheduler-config]. The