kemko/nomad
1
0
Fork 0
mirror of https://github.com/kemko/nomad.git synced 2026-01-01 16:05:42 +03:00
Code Issues Packages Projects Releases Wiki Activity

variables: limit rekey eval to half the nack timeout (#15102)

Browse Source 
In order to limit how much the rekey job can monopolize a scheduler worker, we
limit how long it can run to 1min before stopping work and emitting a new
eval. But this exactly matches the default nack timeout, so it'll fail the eval
rather than getting a chance to emit a new one.

Set the timeout for the rekey eval to half the configured nack timeout.
This commit is contained in:
Tim Gross
2022-11-01 16:50:50 -04:00
committed by GitHub
parent 6b2da83f6a
commit 18cb9c76f2
2 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.changelog/15102.txt Normal file
View File

@@ -0,0 +1,3 @@
```release-note:bug
variables: Fixed a bug where a long-running rekey could hit the nack timeout
```

2
nomad/core_sched.go
View File

@@ -1053,7 +1053,7 @@ func (c *CoreScheduler) rotateVariables(iter memdb.ResultIterator, eval *structs
//
// Instead, we'll rate limit RPC requests and have a timeout. If we still
// haven't finished the set by the timeout, emit a new eval.
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Minute)
ctx, cancel := context.WithTimeout(context.Background(), c.srv.GetConfig().EvalNackTimeout/2)
defer cancel()
limiter := rate.NewLimiter(rate.Limit(100), 100)