Commit graph

534 commits

Author SHA1 Message Date
Silvan Mosberger
490d46f044
Merge pull request #148315 from hercules-ci/nixos-evalModules-legacy-cleanup
NixOS/evalModules legacy cleanup
2021-12-07 18:30:52 +01:00
Artturin
60422ba2ea nixos/test-driver: add 10ms delay to send_key
attempt to fix https://github.com/NixOS/nixpkgs/issues/147294
2021-12-03 20:04:56 +02:00
Robert Hensing
78ada83361 nixos/eval-config: Deprecate extraArgs and check parameters 2021-12-02 18:23:43 +00:00
Robert Hensing
0b5aea2b27 nixos/eval-config: Remove a rec 2021-12-02 18:23:43 +00:00
Robert Hensing
59c4a35aab nixos/eval-config: Avoid evalModules args and check parameters 2021-12-02 18:23:43 +00:00
Jacek Galowicz
58371472fe
Merge pull request #146512 from DeterminateSystems/better-visibility
nixos/test-driver: more context when step finishes, give more functions nested labels
2021-12-02 09:26:21 +00:00
Jacek Galowicz
57dc1085dc
Merge pull request #146271 from DeterminateSystems/better-timeouts
nixos/test-driver: add execute_with_timeout, add (functional) timeouts to more functions
2021-12-01 19:01:05 +00:00
Cole Helbling
af765f3abd nixos/test-driver: give more functions nested labels
This will make it easier to trace through the test execution without
having to scroll through the entire kernel output.
2021-12-01 10:29:56 -08:00
Cole Helbling
c6ee63259a nixos/test-driver: more context when step finishes
When displaying the amount of time some step took, with no other
context, it becomes nigh impossible (especially in longer tests) to see
when specific steps finished.
2021-12-01 10:29:56 -08:00
Robert Hensing
8a129f8cf0
Merge pull request #144094 from hercules-ci/nixos-specialisations-use-extendModules
nixos/specialisation: Rephrase in terms of extendModules, noUserModules
2021-12-01 11:03:36 +01:00
Martin Schwaighofer
af180d554b qemu, runInLinuxVM: change default cpu to qemu64
The flag -cpu max leaves QEMU 6.1.0 stuck on some systems,
for example when /dev/kvm is not read-writable.
This does not happen with -cpu qemu64.

Getting stuck like that is a regression in 6.1.0 not yet present in 6.0.0
and should be fixed with 6.2.0 according to early testing with rc1.

We should consider reverting this change when we merge QEMU 6.2.0.
See #146526.

fixes #141596
2021-11-30 13:06:22 +00:00
talyz
e8cc900eae
make-disk-image: Make additionalPaths work with Nix 2.4
The `nix` command is marked as experimental since 2.4, so an extra
flag is required to unlock it.
2021-11-29 21:11:15 +01:00
Ryan Burns
2175b157ac treewide: refactor isi686 && isx86_64 -> isx86 2021-11-20 17:50:41 -08:00
Patrick Hilhorst
1334a62539
test-driver.py: directly import pathlib.Path 2021-11-20 01:37:08 +01:00
Patrick Hilhorst
9e71014ede
test-driver.py: always export single machine as 'machine' 2021-11-19 23:02:30 +01:00
Patrick Hilhorst
a8f693ed48
test-driver.py: fix weird non-pythonism 2021-11-19 22:28:32 +01:00
Cole Helbling
e62b8020f3 nixos/test-driver: add (functional) timeouts to more functions
A retry timeout doesn't really help if the thing it's retrying may block
forever.
2021-11-17 16:11:10 -08:00
Cole Helbling
363d7f3ae8 nixos/test-driver: add timeout parameter to execute 2021-11-17 16:11:03 -08:00
Artturin
83f892c511 nixos/lib: add /usr to pathsNeededForBoot
if /usr is split from root then it has to be mounted at boot
See https://github.com/NixOS/nixpkgs/issues/146115
2021-11-15 16:28:22 +02:00
Jacek Galowicz
2ba0732988
Merge pull request #142747 from helsinki-systems/fix/test-runner-execute
nixos/test-runner: Fix execute() flakiness
2021-11-02 14:51:50 +01:00
Robert Hensing
b5681a7a40 nixos/specialisation: Rephrase in terms of extendModules, noUserModules
By using the new extendModules function to produce the specialisations,
we avoid reimplementing the eval-config.nix logic in reverse and fix
cross compilation support for specialisations in the process.
2021-11-01 10:59:47 +01:00
Robert Hensing
0b0d2637fc Revert "nixos/specialisation: Rephrase in terms of extendModules, noUserModules"
This reverts commit e2bea4427b.

While this commit was probably fine, I want to be conservative
with changes right before the release branch-off.
So far the extendModules commits have been adding and refactoring
expressions that did not affect derivation hashes, whereas this
commit changes the module ordering. I will open a separate PR for
it.
2021-11-01 09:41:55 +01:00
Robert Hensing
e2bea4427b nixos/specialisation: Rephrase in terms of extendModules, noUserModules 2021-11-01 09:40:16 +01:00
Robert Hensing
22584ce667 nixos/eval-config.nix: Expose type
Also works for (pkgs.nixos {}).type.
2021-11-01 09:38:49 +01:00
Robert Hensing
56c4f9d005 nixosTest: Fix infinite recursion involving hasContext testScript when useNixStoreImage = true
The involved test was nixosTests.nextcloud.basic21.

It has a testScript that is strict in nodes.nextcloud.config.system.build.vm,
in assertions about imagemagick being in the system closure.

The recursion was introduced in 329a4461a7 from
https://github.com/NixOS/nixpkgs/pull/140792
2021-10-31 18:25:19 +01:00
talyz
46f7521144 make-disk-image: Add an argument to only build a Nix store image
Add the `onlyNixStore` argument which enables building images
containing the contents of the Nix store at the root, instead of a
full system.
2021-10-28 12:55:01 +02:00
talyz
329a4461a7 nixos/testing-python: Copy test script derivations to nodes
Make sure the all derivations referenced by the test script are
available on the nodes. Accessing these derivations works just fine
without this change when using 9p to mount the host's store, but when
an image is built (virtualisation.buildRootImage), the dependencies
need to be copied to the image. We don't want to copy the script
itself, though, since that would trigger unnecessary image rebuilds.
2021-10-28 12:55:01 +02:00
talyz
87d8eec069 make-disk-image: Add additionalPaths argument
Add an argument which makes it possible to add additional paths to the
image's nix store.
2021-10-28 12:55:01 +02:00
talyz
398a73ac98 make-disk-image: Add copyChannel argument
Add a copyChannel argument which controls whether the current source
tree will be made available as a nix channel in the image or
not. Previously, it always was. Making it available is useful for
interactive use of nix utils, but changes the hash of the image when
the sources are updated.
2021-10-28 12:55:01 +02:00
talyz
56c5efa25b make-disk-image: Reintroduce the installBootLoader argument
installBootLoader was removed in
f1708a9d7d79e2bf2961fc648625578b23b3460f; probably by mistake, since
only the logic was removed, not the argument.
2021-10-28 12:55:01 +02:00
Janne Heß
1640359f33
nixos/test-runner: Fix execute() flakiness
Instead of using the magic string, we now just base64-encode everything
and check for a newline.
2021-10-28 11:51:20 +02:00
Janne Heß
1ed5606d8a
nixos/test-runner: Fix thread cleanup 2021-10-24 15:11:46 +02:00
K900
9611041ef5 nixos/lib/test-driver: set defaults in __init__ instead of class level
Just a minor code style nit to avoid potential footguns
2021-10-19 16:29:19 +03:00
K900
a874235dff nixos/lib/test-driver: clean up threads correctly
The current implementation just forks off a thread to read
QEMU's stdout and lets it exist forever. This, however,
makes the interpreter shutdown racy, as the thread could
still be running and writing out buffered stdout when the
main thread exits (and since it's using the low level API,
the worker thread does not get cleaned up by the atexit hooks
installed by `threading`, either). So, instead of doing that,
let's create a real `threading.Thread` object, and also
explicitly `join` it along with the other stuff when cleaning up.
2021-10-19 16:23:51 +03:00
Domen Kožar
8fdc8fe5ce
Merge pull request #142040 from pennae/optimize-module-docs
make-options-docs: don't sort the options XML file
2021-10-18 16:38:32 -05:00
pennae
24eb353907 make-options-docs: don't sort the options XML file
we need the file itself as a dependency for the docbook build, but we don't need
it to be properly sorted at the nix level. push the sort out to a python script
instead to save eval time. on the machine used to write this `nix-instantiate
<nixos/nixos> -A system` went down from 7.1s to 5.4s and GC heap size decreased
by 50MB (or 70MB max RSS).
2021-10-18 03:45:33 +02:00
Robert Hensing
c247827654 nixosTest: Provide system.build.networkConfig 2021-10-17 23:56:52 +02:00
Janne Heß
540dc908ca
nixos/test-runner: Print exceptions that happen 2021-10-14 00:15:53 +02:00
Kim Lindberger
4e97bedf93
Merge pull request #112823 from deviant/nixos-boot-paths
nixos/boot: add /var/lib/nixos to pathsNeededForBoot
2021-10-07 17:38:54 +02:00
Jacek Galowicz
d68f1e43df
Merge pull request #126713 from blaggacao/nixos-test-ref/05-class-ify-test-driver
nixos test ref/05 class ify test driver
2021-10-07 14:37:01 +02:00
Jacek Galowicz
5c666cdf62 Re-RAII-ify the NixOS integration test driver's VLAN class.
We have no usecase for manually/selectively starting or stopping VLANs
in integration tests.
By starting and stopping the VLANs with the constructor and destructor
of VLAN objects, we remove the obligation and complexity to maintain
network lifetime separately.
2021-10-05 14:38:48 -05:00
Maximilian Bosch
3f63e3ce65 nixos/test-driver: fix graphics for VM
`-nographic` should only be set if no display is available. Otherwise,
tools such as `nixos-build-vms(8)` will never open up a graphical
display.
2021-10-05 14:38:48 -05:00
David Arnold
b0fc9da879 nixos/test/test-driver: Class-ify the test driver
This commit encapsulates the involved domain into classes and
defines explicit and typed arguments where untyped dicts where used.

It preserves backwards compatibility through legacy wrappers.
2021-10-05 14:38:48 -05:00
Naïm Favier
330b1e08b8
nixos/lib/make-options-doc: implement literalDocBook 2021-10-03 17:59:44 +02:00
rnhmjoj
b29c2f97c3
nixos/lib/qemu-flags: rename to qemu-common
The current name is misleading: it doesn't contain cli arguments,
but several constants and utility functions related to qemu.
This commit also removes the use of `with import ...` for clarity.
2021-09-18 16:58:16 +02:00
rnhmjoj
30f0faac22
nixos/qemu: add types to all options 2021-09-18 16:47:11 +02:00
rnhmjoj
c1fd51f490
nixos/qemu: fix several shellcheck issues
- Fix shell quoting issues
- Fix unsafe cd in run-machine-vm script
2021-09-18 16:46:20 +02:00
Matthew Kenigsberg
b00d2a8f3e
nixos/testing: remove unused function runInMachine
runInMachine and runInMachineWithX are not used for any tests and can be
removed
2021-09-08 06:25:20 -05:00
Graham Christensen
076f6e2d94 nixos/make-zfs-image: init
This is a private interface for internal NixOS  use. It is similar
to `make-disk-image` except it is much more opinionated about what
kind of disk image it'll make.

Specifically, it will always create *two* disks:

1. a `boot` disk formatted with FAT in a hybrid GPT mode.
2. a `root` disk which is completely owned by a single zpool.

The partitioning and FAT decisions should make the resulting images
bootable under EFI or BIOS, with systemd-boot or grub.

The root disk's zpools options are highly customizable, including
fully customizable datasets and their options.

Because the boot disk and partition are highly opinionated, it is
expected that the `boot` disk will be mounted at `/boot`. It is
always labeled ESP even on BIOS boot systems.

In order for the datasets to be mounted properly, the `datasets`
passed in to `make-zfs-image` are turned in to NixOS configuration
stored at /etc/nixos/configuration.nix inside the VM.
NOTE: The function accepts a system configuration in the `config`
argument. The *caller* must manually configure the system
in `config` to have each specified `dataset` be represented
by a corresponding `fileSystems` entry.

One way to test the resulting images is with qemu:

```sh
boot=$(find ./result/ -name '*.boot.*');
root=$(find ./result/ -name '*.root.*');

echo '`Ctrl-a h` to get help on the monitor';
echo '`Ctrl-a x` to exit';

qemu-kvm \
    -nographic \
    -cpu max \
    -m 16G \
    -drive file=$boot,snapshot=on,index=0,media=disk \
    -drive file=$root,snapshot=on,index=1,media=disk \
    -boot c \
    -net user \
    -net nic \
    -msg timestamp=on
```
2021-08-25 10:42:35 -04:00
David Arnold
2832896c5b
nixos/test: some test fixes in succession of #125992 2021-08-20 13:38:22 -05:00