This changes much of the make-disk-image.nix logic (and thus most NixOS
image building) to use LKL to set up the target directory structure rather
than a Linux VM. The only work we still do in a VM is less IO-heavy stuff
that while still time-consuming, is less of the overall load. The goal is
to kill more of that stuff, but that will require deeper changes to NixOS
activation scripts and switch-to-configuration.pl, and I don't want to
bite off too much at once.
This makes make-disk-image.nix slightly more consistent with other image
builders we have. Unfortunately I duplicated some code in doing so, but
this is temporary duplication on the path to consolidating everything.
See https://github.com/NixOS/nixpkgs/issues/23052 for more details on that.
I'm also exposing the option in the amazon-image.nix maintainer module.
tune2fs marks the filesystem as clean to prevent resize2fs from
complaining.
But we were invoking it before we mounted the filesystem, so the
counters would increase to 1 and it broke the functionality.
By moving the call after the mount, I have confirmed it works by:
$ nix-build nixos/tests/ec2.nix
cc @rbvermaa @edolstra
- Replace hand-rolled version of nixos-install in make-disk-image by an
actual call to nixos-install
- Required a few cleanups of nixos-install
- nixos-install invokes an activation script which the hand-rolled version
in make-disk-image did not do. We remove /etc/machine-id as that's
a host-specific, impure, output of the activation script
Testing:
nix-build '<nixpkgs/nixos/release.nix>' -A tests.installer.simple passes
Also tried generating an image with:
nix-build -E 'let
pkgs = import <nixpkgs> {};
lib = pkgs.lib;
nixos = import <nixpkgs/nixos> {
configuration = {
fileSystems."/".device = "/dev/disk/by-label/nixos";
boot.loader.grub.devices = [ "/dev/sda" ];
boot.loader.grub.extraEntries = '"''"'
menuentry "Ubuntu" {
insmod ext2
search --set=root --label ubuntu
configfile /boot/grub/grub.cfg
}
'"''"';
};
};
in import <nixpkgs/nixos/lib/make-disk-image.nix> {
inherit pkgs lib;
config = nixos.config;
diskSize = 2000;
partitioned = false;
installBootLoader = false;
}'
Then installed the image:
$ sudo df if=./result/nixos.img of=/dev/sdaX bs=1M
$ sudo resize2fs /dev/disk/by-label/nixos
$ sudo mount /dev/disk/by-label/nixos /mnt
$ sudo mount --rbind /proc /mnt/proc
$ sudo mount --rbind /dev /mnt/dev
$ sudo chroot /mnt /nix/var/nix/profiles/system/bin/switch-to-configuration boot
[ … optionally do something about passwords … ]
and successfully rebooted to that image.
Was doing all this from inside a Ubuntu VM with a single user nix install.
We now generate a qcow2 image to prevent hitting Hydra's output size
limit. Also updated /root/user-data -> /etc/ec2-metadata/user-data.
http://hydra.nixos.org/build/33843133
Previously this was done in three derivations (one to build the raw
disk image, one to convert to OVA, one to add a hydra-build-products
file). Now it's done in one step to reduce the amount of copying
to/from S3. In particular, not uploading the raw disk image prevents
us from hitting hydra-queue-runner's size limit of 2 GiB.
This prevents seeing lots of warnings about missing hashes/sizes in the
database when running "nix-store --verify --check-contents" for the
first time.
The EBS and S3 (instance-store) AMIs are now created from the same
image. HVM instance-store AMIs are also generated.
Disk image generation has been factored out into a function
(nixos/lib/make-disk-image.nix) that can be used to build other kinds
of images.
