From e23e366e8a502df8e2300504843103759ff5066a Mon Sep 17 00:00:00 2001 From: Luiz Aoqui Date: Tue, 31 Jan 2023 15:04:12 -0500 Subject: [PATCH] docs: add upgrade notice for batch GC changes (#15985) --- .../content/docs/upgrade/upgrade-specific.mdx | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/website/content/docs/upgrade/upgrade-specific.mdx b/website/content/docs/upgrade/upgrade-specific.mdx index 13020abe5..445caa0eb 100644 --- a/website/content/docs/upgrade/upgrade-specific.mdx +++ b/website/content/docs/upgrade/upgrade-specific.mdx @@ -84,6 +84,36 @@ The metric `nomad.nomad.broker.total_blocked` has been changed to leader's broker, and this is easily confused with the unrelated evaluation status `"blocked"` in the Nomad API. +#### Garbage collection of evaluations and allocations for batch job + +Versions prior to 1.5.0 only delete evaluations and allocations of batch jobs +that are explicitly stopped which can lead to unbounded memory growth of Nomad +when the batch job is executed multiple times. + +Nomad 1.5.0 introduces a new server configuration +[`batch_eval_gc_threshold`](/nomad/docs/configuration/server#batch_eval_gc_threshold) +to control how allocations and evaluations for batch jobs are collected. + +The default threshold is `24h`. If you need to access completed allocations for +batch jobs that are older than 24h you must increase this value when upgrading +Nomad. + +## Nomad 1.4.4, 1.3.9 + +#### Garbage collection of evaluations and allocations for batch job + +Versions prior to 1.4.4 and 1.3.9 only delete evaluations and allocations of +batch jobs that are explicitly stopped which can lead to unbounded memory +growth of Nomad when the batch job is executed multiple times. + +Nomad 1.4.4 and 1.3.9 introduces a new server configuration +[`batch_eval_gc_threshold`](/nomad/docs/configuration/server#batch_eval_gc_threshold) +to control how allocations and evaluations for batch jobs are collected. + +The default threshold is `24h`. If you need to access completed allocations for +batch jobs that are older than 24h you must increase this value when upgrading +Nomad. + ## Nomad 1.4.0 #### Possible Panic During Upgrades