Commit Graph

565 Commits

Author SHA1 Message Date
Preetha Appan
fc9c87c032 Remove unnecessary usage of alloc.Resource 2019-01-10 16:36:47 -06:00
Mahmood Ali
d19245fa7b appease linter 2019-01-08 10:58:49 -05:00
Alex Dadgar
19e67a0916 Test recovery 2019-01-07 14:49:41 -08:00
Preetha
44cc76c4a3 Merge pull request #4881 from hashicorp/f-device-preemption
Device preemption
2018-12-11 18:34:19 -06:00
Preetha Appan
e7162e8bd8 Early continue after meeting needed count
Also adds another optimization that filters out un-needed allocations
as a final filtering step
2018-12-11 10:12:18 -06:00
Preetha Appan
3921793030 Score combinations of allocs from multiple devices for preemption 2018-12-07 18:35:47 -06:00
Alex Dadgar
0953d913ed Deprecate IOPS
IOPS have been modelled as a resource since Nomad 0.1 but has never
actually been detected and there is no plan in the short term to add
detection. This is because IOPS is a bit simplistic of a unit to define
the performance requirements from the underlying storage system. In its
current state it adds unnecessary confusion and can be removed without
impacting any users. This PR leaves IOPS defined at the jobspec parsing
level and in the api/ resources since these are the two public uses of
the field. These should be considered deprecated and only exist to allow
users to stop using them during the Nomad 0.9.x release. In the future,
there should be no expectation that the field will exist.
2018-12-06 15:09:26 -08:00
Preetha Appan
4afd512f45 use structured logging everywhere consistently 2018-12-03 08:31:41 -06:00
Preetha Appan
e023b367fa addresses some code clarity review comments 2018-11-27 11:02:06 -06:00
Mahmood Ali
e7257fe5be Simplify map count update logic
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-11-27 10:03:11 -06:00
Mahmood Ali
026f976761 code review suggestion
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-11-27 09:59:57 -06:00
Preetha Appan
e8088e404b Fix formatting 2018-11-16 20:45:52 -06:00
Preetha Appan
7f2826097d Fix preemption logic bug, need to group allocations by device first.
This ensures that the set of allocations chosen for preemption all share
the same device where ID is <vendor/type/device>
2018-11-16 20:32:10 -06:00
Danielle Tomlinson
3d0a45f6e5 scheduler: Add is_set/is_not_set constraints
This adds constraints for asserting that a given attribute or value
exists, or does not exist. This acts as a companion to =, or !=
operators, e.g:

```hcl
constraint {
        attribute = "${attrs.type}"
        operator  = "!="
        value     = "database"
}

constraint {
        attribute = "${attrs.type}"
        operator  = "is_set"
}
```
2018-11-15 11:00:32 -08:00
Preetha Appan
813143fee5 fix linting 2018-11-15 12:27:32 -06:00
Preetha Appan
9dfc27915c Initial implementation of device preemption 2018-11-15 11:09:26 -06:00
Danielle Tomlinson
0925bfe618 scheduler: Allow comparisons of nil values
This commit allows the ConstraintChecker to test values that do not exist.
This is useful when wanting to _exclude_ given nodes from executing a
job, for example, if you wanted to give canary nodes an attribute, and
not run critical services on them, you may specify something like the
below, but not want to tag all other nodes with the inverse.

```hcl
constraint {
  attribute = "${node.attr.canary}
  operator = "!="
  value = "1"
}
```

This also requires all constraint checkers to allow for nil target
values, as they will no longer be short circuited by resolving a target.
2018-11-13 13:36:51 -08:00
Alex Dadgar
895fdb79f1 Merge pull request #4867 from hashicorp/b-deployment-progress-deadline
Blocked evaluation fixes
2018-11-13 10:29:03 -08:00
Preetha Appan
3a2d5f0178 blank line 2018-11-12 15:50:14 -06:00
Preetha Appan
9f7dfb17a5 Fix logic bug in tracking sum of matched affinity weights
We need to track the sum of matching weights per device, but only
change the final return value if its the highest scoring choice
2018-11-12 15:06:45 -06:00
Preetha Appan
eda98fb070 Normalize scores correctly 2018-11-08 17:01:58 -06:00
Preetha Appan
66670c0f02 Fixes device scheduling unit tests
Also changes the logic for score when there is more than one task
requesting a device. Since inter task affinities are already normalized,
we take the average of the scores across tasks.
2018-11-08 10:31:19 -06:00
Alex Dadgar
2c6ea24f3c fix test 2018-11-07 11:59:24 -08:00
Alex Dadgar
ad4c26a1e3 review comments 2018-11-07 11:31:52 -08:00
Alex Dadgar
a36b682cf4 review comments 2018-11-07 10:33:22 -08:00
Alex Dadgar
ea5fd244f6 allocs fit checks if devices get oversubscribed 2018-11-07 10:33:22 -08:00
Alex Dadgar
dcbf9bdbab Split device accounter and allocator 2018-11-07 10:32:03 -08:00
Alex Dadgar
7ab3ce4bde affinities 2018-11-07 10:32:03 -08:00
Alex Dadgar
77ad27de60 assign devices 2018-11-07 10:32:03 -08:00
Alex Dadgar
0397cbc8d1 Duplicate blocked evals cancelling improved
The old logic for cancelling duplicate blocked evaluations by job id had
the issue where the newer evaluation could have additional node classes
that it is (in)eligible for that we would not capture. This could make
it such that cluster state could change such that the job would make
progress but no evaluation was unblocked.
2018-11-07 10:08:23 -08:00
Preetha Appan
ea5d8cb625 update preemption tests to use new node resource structs
also includes a fix to remove unnecessary subtraction of network mbits
2018-11-02 17:59:53 -05:00
Preetha
6aa0c7ffa9 Merge pull request #4794 from hashicorp/f-preemption-systemjobs
Preemption for system jobs
2018-11-02 16:28:06 -05:00
Preetha Appan
97cf4e1d7a Address more minor code review feedback 2018-11-02 16:26:34 -05:00
Preetha Appan
c49a3e20c5 Fix test setup 2018-11-02 16:06:25 -05:00
Preetha Appan
1380acbf56 dereference safely 2018-11-02 15:58:59 -05:00
Preetha Appan
8235919158 Fix static port preemption to be device aware 2018-11-02 13:07:24 -05:00
Preetha Appan
0015095b57 Handle static port preemption when there are multiple devices
Also added test case
2018-11-02 09:09:50 -05:00
Preetha Appan
06ad182026 Plumb alloc resource cache in a few more places.
also removed now unused method
2018-11-01 16:44:43 -05:00
Preetha Appan
3ad7b3fb5b More review comments 2018-11-01 16:36:11 -05:00
Preetha Appan
993b6a272a Cleaner way to exit early, and fixed a couple more places reading from alloc.Resources 2018-11-01 16:15:58 -05:00
Preetha Appan
22d156f5a0 review comments 2018-11-01 12:01:59 -05:00
Preetha Appan
4a35d62887 Fix formatting of allocation score metrics 2018-10-30 12:03:23 -05:00
Preetha Appan
17344a7f3a Introduce interface with multiple implementations for resource distance 2018-10-30 11:06:32 -05:00
Preetha Appan
35635ba39a refactor preemption code to use method recievers and setters for common fields 2018-10-30 11:06:32 -05:00
Preetha Appan
4cc21fbcfa more minor cleanup 2018-10-30 11:06:32 -05:00
Preetha Appan
21432d69dc More style and readablity fixes from review 2018-10-30 11:06:32 -05:00
Preetha Appan
655689a743 Preempted allocations should be removed from proposed allocations 2018-10-30 11:06:32 -05:00
Preetha Appan
a960cced8c comments 2018-10-30 11:06:32 -05:00
Preetha Appan
9f35923db0 fix end to end scheduler test to use new resource structs correctly 2018-10-30 11:06:32 -05:00
Preetha Appan
24b393483a Modify preemption code to use new style of resource structs 2018-10-30 11:06:32 -05:00