Ensure that the index processed by the client is at least as new as the last one processed. (#18269)

Ensure that the index processed by the client is at least as new as the last index processed so that stale data does not impact the running allocations.
This commit is contained in:
stswidwinski
2023-08-25 18:54:58 -04:00
committed by GitHub
parent da830b1046
commit f25480c9e9
2 changed files with 15 additions and 0 deletions

3
.changelog/18269.txt Normal file
View File

@@ -0,0 +1,3 @@
```release-note:bug
client: Ignore stale server updates to prevent GCing allocations that should be running
```

View File

@@ -2360,6 +2360,18 @@ OUTER:
default:
}
// We have not received any new data, or received stale data. This may happen in
// an array of situations, the worst of which seems to be a blocking request
// timeout when the scheduler which we are contacting is newly added or recovering
// after a prolonged downtime.
//
// For full context, please see https://github.com/hashicorp/nomad/issues/18267
if resp.Index <= req.MinQueryIndex {
c.logger.Debug("Received stale allocation information. Retrying.",
"index", resp.Index, "min_index", req.MinQueryIndex)
continue OUTER
}
// Filter all allocations whose AllocModifyIndex was not incremented.
// These are the allocations who have either not been updated, or whose
// updates are a result of the client sending an update for the alloc.