Commit Graph

20940 Commits

Author SHA1 Message Date
Mahmood Ali
43549b46fc driver/exec: set soft memory limit
Linux offers soft memory limit:
https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/memory.html#soft-limits
, and
https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html?highlight=memory.low
.

We can set soft memory limits through libcontainer
`Resources.MemoryReservation`: https://pkg.go.dev/github.com/opencontainers/runc@v0.1.1/libcontainer/configs#Resources
2021-03-30 16:55:58 -04:00
Mahmood Ali
750e665a1d oversubscription: e2e tests! 2021-03-30 16:55:58 -04:00
Mahmood Ali
5e3fbd5774 oversubscription: driver/exec to honor MemoryMaxMB 2021-03-30 16:55:58 -04:00
Mahmood Ali
b1ff06fd19 oversubscription: docker to honor MemoryMaxMB values 2021-03-30 16:55:58 -04:00
Mahmood Ali
b383f92188 oversubscription: set the linux memory limit
Use the MemoryMaxMB as the LinuxResources limit. This is intended to ease
drivers implementation and adoption of the features: drivers that use
`resources.LinuxResources.MemoryLimitBytes` don't need to be updated.

Drivers that use NomadResources will need to updated to track the new
field value. Given that tasks aren't guaranteed to use up the excess
memory limit, this is a reasonable compromise.
2021-03-30 16:55:58 -04:00
Mahmood Ali
18e1a599ed oversubscription: add memory_max to hclv1
Allow specifying the `memory_max` field in HCL under the resources block.

Though HCLv1 is deprecated, I've updated them to ease our testing.
2021-03-30 16:55:58 -04:00
Mahmood Ali
7df90da9c8 oversubscription: adds CLI and API support
This commit updates the API to pass the MemoryMaxMB field, and the CLI to show
the max set for the task.

Also, start parsing the MemoryMaxMB in hcl2, as it's set by tags.

A sample CLI output; note the additional `Max: ` for "task":

```
$ nomad alloc status 96fbeb0b
ID                  = 96fbeb0b-a0b3-aa95-62bf-b8a39492fd5c
[...]

Task "cgroup-fetcher" is "running"
Task Resources
CPU        Memory         Disk     Addresses
0/500 MHz  32 MiB/20 MiB  300 MiB

Task Events:
[...]

Task "task" is "running"
Task Resources
CPU        Memory          Disk     Addresses
0/500 MHz  176 KiB/20 MiB  300 MiB
           Max: 30 MiB

Task Events:
[...]
```
2021-03-30 16:55:58 -04:00
Mahmood Ali
80faab0f79 oversubscription: Add MemoryMaxMB to internal structs
Start tracking a new MemoryMaxMB field that represents the maximum memory a task
may use in the client. This allows tasks to specify a memory reservation (to be
used by scheduler when placing the task) but use excess memory used on the
client if the client has any.

This commit adds the server tracking for the value, and ensures that allocations
AllocatedResource fields include the value.
2021-03-30 16:55:58 -04:00
Tim Gross
14568b3e00 deps: bump gopsutil to v3.21.2 2021-03-30 16:02:51 -04:00
Tim Gross
f1645dd53a changelog: add section for backports from 1.1.0 to 1.0.5
* Fixed order
* Added missing changlog entry for GH-10145
* Fixed misclassified entry for GH-10158
2021-03-30 15:53:17 -04:00
Bryce Kalow
dee1664c83 fix(circle): change website branch filter to main (#10261) 2021-03-30 10:29:25 -05:00
Seth Hoenig
7da9ece876 Merge pull request #10243 from apollo13/issue10239
Automatically populate `CONSUL_HTTP_ADDR` for connect native tasks in host networking mode.
2021-03-30 09:00:17 -05:00
Nick Ethier
0a25e153c7 Merge pull request #10203 from hashicorp/f-cpu-cores
Reserved Cores [1/4]: Structs and scheduler implementation
2021-03-29 14:05:54 -04:00
Zachary Shilton
5ca4acff06 website: use global featured-slider component (#10254)
* website: use global featured-slider component

* website: delete unused local featured-slider
2021-03-29 11:37:48 -04:00
Bryce Kalow
9dc47fd8b7 website: add should-build script (#10194) 2021-03-29 10:06:20 -05:00
Dave May
5e6cb151c5 debug: Remove extra linefeed in monitor.log (#10252) 2021-03-29 09:22:27 -04:00
Florian Apolloner
f21ab14690 Automatically populate CONSUL_HTTP_ADDR for connect native tasks in host networking mode. Fixes #10239 2021-03-28 14:34:31 +02:00
Chris Baker
b72bafaaed Merge pull request #10125 from hashicorp/docs-autoscaler-409-plugins
autoscaler plugin docs
2021-03-28 07:07:24 -05:00
Luiz Aoqui
233dba8bc2 docs: minor updates on autoscaling plugin guides 2021-03-26 16:55:24 -04:00
Tim Gross
3d3980eb64 redirect from HTTP root to UI should include query params
The OTT feature relies on having a query parameter for a one-time token which
gets handled by the UI. We need to make sure that query param is preserved
when redirecting from the root URL to the `/ui/` URI.
2021-03-26 14:54:41 -04:00
Tim Gross
8dc695987c docs: note that Connect requires a hard-coded port
Co-authored-by: Kris Hicks <khicks@hashicorp.com>
2021-03-26 14:43:29 -04:00
Shishir
12fc27cc08 Update containerd task driver docs. (#10244) 2021-03-26 14:42:27 -04:00
Drew Bailey
fe0716ed43 e2e license smoke test (#10242) 2021-03-26 13:21:47 -04:00
Buck Doyle
3d9464023d Update to v4 of Ember Power Select (#10226)
This closes #10146.

Because of cibernox/ember-power-select#1203, which documents
the current impossibility of attaching test selectors to a
PowerSelect invocation, this uses test selectors on parent
containers instead, occasionally adding wrappers when needed.
I chose to leave the existing test selectors in the hopes that
we can return to using them eventually, but I could easily
remove them if it seems like extra noise now.

Presumably for the same reason, @class no longer works, so
this adjusts the scoping of global search CSS to preserve the style
of the search control.

I also included an update to the latest version of
ember-test-selectors, since we were far behind and I tried
that before finding the aforelinked issue.

Finally, this replaces ember-cli-uglify with ember-cli-terser to address
production build failures as described at ember-cli/ember-cli#9290.
2021-03-26 08:55:12 -05:00
Mahmood Ali
274e795be5 Merge pull request #10145 from hashicorp/b-periodic-init-status
periodic: always reset periodic children status
2021-03-26 09:19:08 -04:00
James Rasell
db3761782b Merge pull request #10230 from hashicorp/f-add-allocation-terminal-status-funcs-api
api: add Allocation client and server terminal status funcs.
2021-03-26 08:03:37 +01:00
Mahmood Ali
d81df3dd1a dispatched parameterized job should clear status too 2021-03-25 15:14:21 -04:00
Drew Bailey
15cf77ad66 e2e allow setting an enterprise license environment variable (#10233)
* allow setting an enterprise license environment variable

* update comment

* address pr comments
2021-03-25 14:35:55 -04:00
Taylor Dolezal
554f69394c Change enable to enabled 2021-03-25 14:10:54 -04:00
Michael Lange
c4a7728e03 Merge pull request #10208 from hashicorp/f-ui/use-new-chart-stuff
UI: Showed host reservations and allocation utilization by task on existing stats charts
2021-03-25 09:18:42 -07:00
Mahmood Ali
032945b718 Add a test for parameterized summary counts 2021-03-25 11:27:09 -04:00
Mahmood Ali
15fd4f3a8e periodic: always reset periodic children status
Fixes a bug where Nomad reports negative or incorrect running children
counts for periodic jobs.

The periodic dispatcher derives a child job without reseting the status.
If the periodic job has a `running` status, the derived job will start
as `running` status and transition to `pending`.  Since this is
unexpected transition, the counting in StateStore.setJobSummary gets out of sync and
result in negative/incorrect values.

Note that this only affects periodic jobs after a leader transition.
During the first job registration, the job is added with `pending` or
`""` status. However, after a leader transition, the new leader
repopulates the dispatcher heap with `"running"` status and triggers the
bug.
2021-03-25 11:27:09 -04:00
James Rasell
788f9f23a0 api: add Allocation client and server terminal status funcs. 2021-03-25 08:52:59 +01:00
Michael Lange
c15bf65ef0 Remove errant log line 2021-03-24 15:34:46 -07:00
Michael Lange
d9426df7af Correctly sort tasks in alloc stat tracker so the main task takes precedence 2021-03-24 15:10:46 -07:00
Michael Lange
1d8bc393cf Guard against empty data when determining active linechart data 2021-03-24 15:09:47 -07:00
Michael Lange
ebd91a9a78 Fix line-wrapping issue with chart tooltips 2021-03-24 14:13:30 -07:00
Michael Lange
2b1449ce88 Use more explicit branching of reserved amounts based on metric type 2021-03-24 13:55:18 -07:00
Michael Lange
b009fc1f97 Remove old primary metric 2021-03-24 13:52:52 -07:00
Michael Lange
5191693be9 Update topology info box primary metric 2021-03-24 13:52:52 -07:00
Michael Lange
91521d6256 Updated old tests 2021-03-24 13:52:52 -07:00
Michael Lange
ead08e2ec6 Tests for PrimaryMetric::Task 2021-03-24 13:52:52 -07:00
Michael Lange
e3a1106f9d Tests for PrimaryMetric::Allocation 2021-03-24 13:52:52 -07:00
Michael Lange
2ed5b28805 Tests for PrimaryMetric::Node 2021-03-24 13:52:52 -07:00
Michael Lange
295354262a Remove now unused chartClass arg from LineChart 2021-03-24 13:52:52 -07:00
Michael Lange
17cf4ad20e Improved language around what the client stat charts represent 2021-03-24 13:52:52 -07:00
Michael Lange
7b73460769 New PrimaryMetric::Task component 2021-03-24 13:52:51 -07:00
Michael Lange
745a55bd42 Apply the new multi-line primary metric to the alloc index page 2021-03-24 13:52:51 -07:00
Michael Lange
df116888c8 Memoize the tracker and the series properties to avoid repeat computation
In addition to this computation being wasteful, it introduces a bug
where the allocation on a stats tracker can update twice in one render,
which isn't allowed in Glimmer (ironically, Glimmmer's lack of
auto-memoization introduced the issue).
2021-03-24 13:52:51 -07:00
Michael Lange
4f27bee77c Refactored multi-series allocation variant of the primary metric component 2021-03-24 13:52:51 -07:00