* elk7: 7.11.1 -> 7.16.1
* nixosTests.elk: Improve reliability and compatibility with ELK 7.x
- Use comparisons in jq instead of grepping
- Match for `.hits.total.value` if version >= 7, otherwise it always
passes
- Make curl fail if requests fails
* nixos/filebeat: Add initial module and test
Filebeat is an open source file harvester, mostly used to fetch logs
files and feed them into logstash.
This module can be used instead of journalbeat if used with
`filebeat7` and configured with the `journald` input.
* python3Packages.parsedmarc.tests: Fix breakage
- Don't use the deprecated elasticsearch7-oss package
- Improve jq query robustness and add tracing
* rl-2205: Note the addition of the filebeat service
* elk6: 6.8.3 -> 6.8.21
The latest version includes a fix for CVE-2021-44228.
* nixos/journalbeat: Add a loose dependency on elasticsearch
Avoid unnecssary back-off when elasticsearch is running on the same
host.
On reboots and shutdowns promtail blocks for at least 90 seconds,
because it would still try to deliver log messages for loki, which isn't
possible when the network has already gone down.
Upstreams example unit also uses a ten seconds timeout, something which
has worked pretty well for me as well.
Since the upstream graylogctl script will prefer finding its java
executable based on JAVA_HOME, we now set this instead of PATH in
order to allow it to find the JRE. By setting it conditionally on it
not already being set, we allow selecting a different JRE at runtime.
We also explicitly use openjdk11, which supports the
UseConcMarkSweepGC option which graylog insists on using.
A centralized list for these renames is not good because:
- It breaks disabledModules for modules that have a rename defined
- Adding/removing renames for a module means having to find them in the
central file
- Merge conflicts due to multiple people editing the central file
Until now the startup failed with an error like this:
```
com.github.joschi.jadconfig.ValidationException: Parent path /var/lib/graylog/server for Node ID file at /var/lib/graylog/server/node-id is not a directory
```
This happens since `graylog.service` ensures that `/var/lib/graylog`
exists, however it doesn't take care of the directory for
`cfg.nodeIdFile`.
