Commit Graph

157 Commits

Author SHA1 Message Date
Michael Schurter
eb2a2cd76e connect: add group.service stanza support 2019-07-31 01:04:05 -04:00
Jasmine Dahilig
e31db578e0 add formatting for hcl parsing error messages (#5972) 2019-07-19 10:04:39 -07:00
Mahmood Ali
95adf6127c Add a test for unknown variables 2019-06-17 12:25:43 -04:00
Mahmood Ali
db3241c0df tests: handle unicode matches
naive implementation should focus on ascii characters only
2019-05-21 09:41:23 -04:00
Mahmood Ali
10797ca61f escapingio: handle stalled readers
Handle stalled readers (e.g. network write got stalled), by having
escaping io have a buffer so it looks for escaped characters in the
stream.

This simplifies the implementation considerably, as we can look
for new lines followed by escaped characters directly.

Also, we add a test to ensure that any partial results are flushed to
readers.
2019-05-17 11:58:31 -04:00
Mahmood Ali
e211a993fd escapingio: thread-safe struct for escaped chars
Use a helper struct for capturing escaped characters that's thread safe.
2019-05-17 10:22:24 -04:00
Mahmood Ali
fb0d002eb1 avoid printing counts in tests 2019-05-16 17:07:32 -04:00
Mahmood Ali
0b4bebd3aa Fix typos and comments
Co-Authored-By: Michael Schurter <michael.schurter@gmail.com>
2019-05-16 17:06:03 -04:00
Mahmood Ali
4e7736023a Add a escaping reader that mimics ssh behavior
Adds an escaping reading that mimics ssh handling of input escape
sequences.

The reader parses chunks to look for \n~
2019-05-16 16:22:52 -04:00
Mahmood Ali
a09e3bf1a1 Allow compiling without nvidia integration
nvidia library use of dynamic library seems to conflict with alpine and
musl based OSes.  This adds a `nonvidia` tag to allow compiling nomad
for alpine images.

The nomad releases currently only support glibc based OS environments,
so we default to compiling with nvidia.
2019-04-10 09:19:12 -04:00
Michael Schurter
91d36f9f6a e2e: add NomadAgent and basic client state test
The e2e test code is absolutely hideous and leaks processes and files
on disk. NomadAgent seems useful, but the clientstate e2e tests are very
messy and slow. The last test "Corrupt" is probably the most useful as
it explicitly corrupts the state file whereas the other tests attempt to
reproduce steps thought to cause corruption in earlier releases of
Nomad.
2019-03-21 07:14:34 -07:00
Mahmood Ali
32623ce3ee Support driver config fields being set to nil (#5391)
To pick up https://github.com/hashicorp/hcl2/pull/90
2019-03-05 21:47:06 -05:00
Preetha
43aa7b138b Merge pull request #5350 from hashicorp/b-json-logging-meta
Support json logging for CLI output for agent
2019-02-22 13:40:56 -06:00
Preetha Appan
b3b4d7b9a3 cli Ui implementation that logs to a hclogger
This makes it so any messages output to the UI *after* the agent has started
will be logged in json format correctly
2019-02-19 17:53:14 -06:00
Mahmood Ali
afb0c5ac45 drivers: restore port_map old json support
This ensures that `port_map` along with other block like attribute
declarations (e.g. ulimit, labels, etc) can handle various hcl and json
syntax that was supported in 0.8.

In 0.8.7, the following declarations are effectively equivalent:

```
// hcl block
port_map {
  http = 80
  https = 443
}

// hcl assignment
port_map = {
  http  = 80
  https = 443
}

// json single element array of map (default in API response)
{"port_map": [{"http": 80, "https": 443}]}

// json array of individual maps (supported accidentally iiuc)
{"port_map: [{"http": 80}, {"https": 443}]}
```

We achieve compatbility by using `NewAttr("...", "list(map(string))",
false)` to be serialized to a `map[string]string` wrapper, instead of using
`BlockAttrs` declaration.  The wrapper merges the list of maps
automatically, to ease driver development.

This approach is closer to how v0.8.7 implemented the fields [1][2], and
despite its verbosity, seems to perserve 0.8.7 behavior in hcl2.

This is only required for built-in types that have backward
compatibility constraints.  External drivers should use `BlockAttrs`
instead, as they see fit.

[1] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L216
[2] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L698-L700
2019-02-16 11:37:33 -05:00
Mahmood Ali
941f89e0fe tests: add hcl task driver config parsing tests (#5314)
* drivers: add config parsing tests

Add basic tests for parsing and encoding task config.

* drivers/docker: fix some config declarations

* refactor and document config parse helpers
2019-02-12 14:46:37 -05:00
Michael Schurter
c12bdcdf0b Merge pull request #5297 from hashicorp/b-docker-logging
Docker: Fix logging config parsing
2019-02-11 06:57:52 -08:00
mnachury
a8c97343cf Fix executable check on windows 2019-02-08 16:05:14 -05:00
Michael Schurter
5a8d955236 docker: fix logging config parsing
Fixes
https://groups.google.com/d/topic/nomad-tool/B3Uo6Kns2BI/discussion
2019-02-04 11:07:57 -08:00
Michael Schurter
08864d5ea5 plugins: update hclutils test
The test used old local copies of Docker structs and appeared to be
testing an outdated approach to task config decoding.

Updated to use real Docker structs so we can do end-to-end unit testing
of real Docker task configs.
2019-02-04 11:07:57 -08:00
Michael Schurter
1d17fbc681 simplify hcl2 parsing helper
No need to pass in the entire eval context
2019-02-04 11:07:57 -08:00
Alex Dadgar
d7407a03b4 Update comment on int8ToPtr 2019-01-30 12:23:43 -08:00
Alex Dadgar
bc42873e07 Change types of weights on spread/affinity 2019-01-30 12:20:38 -08:00
Mahmood Ali
b70442a325 drivers: pass logger through driver plugin client
This fixes a panic whenever driver plugin attempts to log a message.
2019-01-25 09:38:41 -05:00
Alex Dadgar
81916181ac nvidia device plugin docs 2019-01-23 10:58:46 -08:00
Michael Schurter
158c74887e goimports until make check is happy 2019-01-23 06:27:14 -08:00
Michael Schurter
0d61ff0fb9 move pluginutils -> helper/pluginutils
I wanted a different color bikeshed, so I get to paint it
2019-01-22 15:50:08 -08:00
Alex Dadgar
95297c608c goimports 2019-01-22 15:44:31 -08:00
Mahmood Ali
b1293a8993 api: move formatFloat function
`helpers.FormatFloat` function is only used in `api`.  Moving it and
marking it as private.  We can re-export it if we find value later.
2019-01-18 15:31:31 -05:00
Michael Schurter
9e4e2d812d Update helper/testtask/testtask_windows.go
Co-Authored-By: dantoml <dani@tomlinson.io>
2019-01-17 18:43:14 +01:00
Danielle Tomlinson
347b4ad247 Expand unix build definition 2019-01-17 18:43:13 +01:00
Danielle Tomlinson
a51aeb224f testtask: Build on windows 2019-01-17 18:43:13 +01:00
Chris Baker
b12e24ec99 Merge branch 'master' of github.com:hashicorp/nomad into f-1157-validate-node-meta-variables 2019-01-09 18:56:49 +00:00
Mahmood Ali
d19245fa7b appease linter 2019-01-08 10:58:49 -05:00
Chris Baker
64f591875d moved interp key regex out to a helper function 2019-01-08 00:11:47 +00:00
Michael Schurter
fc7be6b7f0 client/state: improve upgradeTaskBucket error handling
And add a test
2018-12-19 10:39:27 -08:00
Mahmood Ali
40164b3dc6 Use max 3 precision in displaying floats
When formating floats in `nomad node status`, use a maximum precision of
3.
2018-12-10 12:18:24 -05:00
Michael Schurter
2a90a336cd boltdd: return error on use-after-Close
Return the same error as boltdb instead of panic'ing.
2018-11-15 14:15:37 -08:00
Mahmood Ali
abd2451d52 address review comments 2018-10-30 13:58:52 -04:00
Mahmood Ali
e9fa36f9ef Allow artifacts checksum interpolation
Fixes https://github.com/hashicorp/nomad/issues/4814
2018-10-30 13:24:30 -04:00
Michael Schurter
d71e7666bd ar: fix leader handling, state restoring, and destroying unrun ARs
* Migrated all of the old leader task tests and got them passing
* Refactor and consolidate task killing code in AR to always kill leader
  tasks first
* Fixed lots of issues with state restoring
* Fixed deadlock in AR.Destroy if AR.Run had never been called
* Added a new in memory statedb for testing
2018-10-19 09:45:45 -07:00
Nick Ethier
391005706f fix package references after drivers/base subpackage removed 2018-10-16 16:53:31 -07:00
Nick Ethier
c9f0d2e0b4 driver/raw_exec: port existing raw_exec tests and add some testing utilities 2018-10-16 16:53:31 -07:00
Michael Schurter
de5426124b lots of comment/log fixes 2018-10-16 16:53:30 -07:00
Michael Schurter
c0d1b63b75 wrap boltdb in a write deduplicator
Saves a tiny bit of cpu and some IO. Sadly doesn't prevent all IO on
duplicate writes as the transactions are still created and committed.

$ go test -bench=. -benchmem
goos: linux
goarch: amd64
pkg: github.com/hashicorp/nomad/helper/boltdd
BenchmarkWriteDeduplication_On-4             500           4059591 ns/op           23736 B/op         56 allocs/op
BenchmarkWriteDeduplication_Off-4            300           4115319 ns/op           25942 B/op         55 allocs/op
2018-10-16 16:53:30 -07:00
Michael Schurter
4107afdbf6 reimplement success state for tr hooks and state persistence
splits apart local and remote persistence

removes some locking *for now*
2018-10-16 16:53:29 -07:00
Alex Dadgar
929eb5570a New parser and comparison 2018-10-12 15:25:34 -07:00
oleksii.shyman
63f4fbf273 Introduce nvidia-plugin fingerprinting
- created go-nvml wrapper for fingerprinting
  - added fingerprinting feature to nvidia-plugin
2018-10-03 15:11:56 -07:00
Alex Dadgar
58c889aa94 yamux 2018-09-17 14:22:40 -07:00
Alex Dadgar
40d095fd1a agent + consul 2018-09-13 10:43:40 -07:00