Commit graph

195 commits

Author SHA1 Message Date
Louis Blin
b3f68289df dockerTools.streamLayeredImage: resolve duplicate env vars
For images running on Kubernetes, there is no guarantee on how duplicate
environment variables in the image config will be handled. This seems
to be different from Docker, where the last environment variable value
is consistently selected.

The current code for `streamLayeredImage` was exploiting that assumption
to easily propagate environment variables from the base image, leaving
duplicates unchecked. It should rather resolve these duplicates to
ensure consistent behavior on Docker and Kubernetes.
2021-03-25 23:29:54 +00:00
Louis Blin
aae8588182 dockerTools.buildLayeredImage: support fromImage
It is now possible to pass a `fromImage` to `buildLayeredImage` and
`streamLayeredImage`, similar to what `buildImage` currently supports.

This will prepend the layers of the given base image to the resulting
image, while ensuring that at most `maxLayers` are used. It will also
ensure that environment variables from the base image are propagated
to the final image.
2021-03-23 14:50:42 +00:00
Benjamin Hipple
17a9f368e3
Merge pull request #115793 from lbpdt/feature/docker-tools-layered-image-name-slashes
dockerTools.buildLayeredImage: image names with registry/ prefix
2021-03-11 14:58:26 -05:00
Louis Blin
419a4fa596 dockerTools.buildLayeredImage: image names with registry/ prefix
When using `buildLayeredImage`, it is not possible to specify an image
name of the form `<registry>/my/image`, although it is a valid name.

This is due to derivations under `buildLayeredImage` using that image
name as their derivation name, but slashes are not permitted in that
context.

A while ago, #13099 fixed that exact same problem in `buildImage` by
using `baseNameOf name` in derivation names instead of `name`. This
change does the same thing for `buildLayeredImage`.
2021-03-10 17:44:24 +00:00
Louis Blin
bf56388c92 dockerTools.buildLayeredImage: configurable store root
`stream_layered_image.py` currently assumes that the store root will be
at `/nix/store`, although the user might have configured this
differently. This makes `buildLayeredImage` unusable with stores having
a different root, as they will fail an assertion in the python script.

This change updates that assertion to use `builtins.storeDir` as the
source of truth about where the store lives, instead of assuming
`/nix/store`.
2021-03-10 16:44:53 +00:00
Chris Roberts
00996b5e03 dockerTools: Do not lowercase image tag in buildImage
Closes #115455
2021-03-09 09:06:04 +00:00
Robert Hensing
f3d006c1d4
Merge pull request #115083 from osener/fix-dockertools-on-darwin
dockerTools: fix build on Darwin
2021-03-04 16:50:26 +01:00
Ozan Sener
78f322f2d0 dockerTools: fix build on Darwin
Fixes #110665

Introduced by #109420
2021-03-04 16:17:48 +01:00
Ben Siraphob
98f26993f2 treewide: remove stdenv where not needed 2021-03-04 19:54:50 +07:00
Johan Thomsen
793e77d4e2 dockertools: sort tar-members by name for reproducibility 2021-02-18 09:51:41 +01:00
Pavol Rusnak
90f7338112
treewide: stdenv.lib -> lib 2021-01-24 01:49:49 +01:00
Robert Hensing
9797139cbb dockerTools: typo
Italian.
2021-01-15 14:23:51 +01:00
Milan
0d724ba2fc
dockerTools: fix build (#109420) 2021-01-15 03:56:38 +01:00
Robert Hensing
a31607f904 dockerTools: Don't apologize
Warning about future breaking changes is wrong.

 - It suggests that the maintainers don't value backwards compatibility.
   They do.
 - It implies that other parts of Nixpkgs won't ever break. They will.
 - It implies that a well-defined "public" interface exists. It doesn't.
 - If the reasons above didn't apply, it should have been in the manual
   instead.

Breaking changes will come, especially to the interface. That can be the
only way we can make progress without breaking the image _contents_.

I don't think dockerTools is any different from most of Nixpkgs in
these regards.
2021-01-06 13:02:19 +01:00
Sarah Brofeldt
ffe5ff6009 dockerTools: Test buildLayeredImage with symlinks
This exercises layer creation in face of store path symlinks, ensuring
they are not dereferenced, which can lead to broken layer tarballs
2021-01-04 21:44:47 +01:00
Sarah Brofeldt
08b0d02944 dockerTools: Fix streamLayeredImage for symlinks
When archiving `/nix/store/foo` and `foo` is itself a symlink, we must
not traverse the symlink target, but archive the `foo` symlink itself
2021-01-04 19:47:34 +01:00
Robert Hensing
5cacf0fcec dockerTools: use go.GOARCH as default arch 2020-12-15 02:15:35 -08:00
Terin Stock
8f66dc94a7 dockerTools: normalize arch to GOARCH
Docker (via containerd) and the the OCI Image Configuration imply and
suggest, respectfully, that the architecture set in images matches those
of GOARCH in the Go Language document.

This changeset updates the implimentation of getArch in dockerTools to
return GOARCH values, to satisfy Docker.

Fixes: #106695
2020-12-15 02:14:01 -08:00
Florian Klink
e054694925 dockerTools.binSh: init 2020-12-02 14:57:23 +01:00
Florian Klink
f7ee2706c2 dockerTools.fakeNss: init
This provides a /etc/passwd and /etc/group that contain root and nobody.

Useful when packaging binaries that insist on using nss to look up
username/groups (like nginx).

The current nginx example used the `runAsRoot` parameter to setup
/etc/group and /etc/passwd (which also doesn't exist in
buildLayeredImage), so we can now just use fakeNss there and use
buildLayeredImage.
2020-12-02 14:56:07 +01:00
edef
de88771a7d nix-prefetch-docker: output informational messages to stderr
Informational messages belong on stderr, not on stdout and intermixed
with structured output for programmatic use.

Change-Id: I34d094d04460494e9ec8953db7490f4e2292d959
2020-12-01 16:58:29 +00:00
Graham Christensen
bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
adisbladis
6847a5c12d
referencesByPopularity: Don't use buildPackages at call-site
It's cleaner to reference buildPackages directly in
build-support/docker directly.
2020-11-24 02:55:40 +01:00
adisbladis
b7b22c5814
dockerTools: Always cross compile for another arch in the cross example
The example fails to build on aarch64, so lets cross build for gnu64.
2020-11-20 12:57:58 +01:00
adisbladis
5357abf49a
dockerTools: Add cross example 2020-11-19 18:13:21 +01:00
adisbladis
8a3b33baed
dockerTools: Set correct architecture when cross compiling 2020-11-19 18:13:21 +01:00
adisbladis
4313ac6b29
dockerTools.buildLayeredImage: Fix cross compilation 2020-11-19 14:32:22 +01:00
Samuel Gräfenstein
0950c98059 docker: add final newline to detjson.py 2020-11-09 11:39:28 +00:00
Michael
24c5059468 docker: Fix broken pipe on docker layer creation
Add `-p` to the `tee` command to avoid exiting on breaking pipe due to
tarsum finishing before tar which creating docker layers.
2020-11-03 12:33:11 +01:00
Robert Hensing
11b0106cf9 dockerTools: add isExe to streamLayeredImage result 2020-10-05 10:48:17 +02:00
Utku Demir
ae82f81bfa
dockerTools.streamLayeredImage: Store the customisation layer as a tarball
This fixes as issue described here[1], where permissions set by 'extraCommands'
were ignored by Nix.

[1] https://github.com/NixOS/nixpkgs/pull/91084#issuecomment-669834938
2020-09-04 16:53:23 +12:00
Johan Thomsen
f5db415e2f nixos/tests/dockerTools: add test for running non-root containers with buildLayeredImage
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-07-31 10:14:07 +02:00
Johan Thomsen
9f86685cc7 dockerTools: fix permissions on /nix/store 2020-07-30 16:20:50 +02:00
Robert Hensing
8c0459f611 dockerTools: Always set imageTag attribute
The image tag can be specified or generated from the output hash.
Previously, a generated tag could be recovered from the evaluated
image with some string operations.

However, with the introduction of streamLayeredImage, it's not
feasible to compute the generated tag yourself.

With this change, the imageTag attribute is set unconditionally,
for the buildImage, buildLayeredImage, streamLayeredImage functions.
2020-07-11 16:58:25 +02:00
Jos van Bakel
66737168a4
dockerTools.buildLayeredImage: fix created=now 2020-07-09 09:34:18 +02:00
Utku Demir
06db331922
dockerTools: Verify nix-store contents on buildLayeredImage test 2020-07-06 16:59:58 +12:00
Utku Demir
2ae2186dfd
dockerTools.streamLayeredImage: Correctly copy hardlinks 2020-07-06 16:42:03 +12:00
Utku Demir
cc46362929
dockerTools: Support files directly under /nix/store
Also makes sure that the files inside a layer added in a sorted order
to make the results more deterministic.
2020-07-04 22:00:57 +12:00
Utku Demir
fa8f2bf34f
stream_layered_image: Use pathlib module for directory traversal 2020-06-22 15:11:04 +12:00
Utku Demir
16199521ea
stream_layered_image: Always set uid and gid to root 2020-06-21 12:35:39 +12:00
Utku Demir
fe71abfc12
stream_layered_image: Clarify assertion failure 2020-06-21 12:35:39 +12:00
Utku Demir
87a538e149
stream_layered_image: Use more descriptive variable name 2020-06-21 12:35:39 +12:00
Utku Demir
4bff5b7f3d
dockerTools: Properly quote a shell variable 2020-06-21 12:35:39 +12:00
Utku Demir
f1d7a09146
stream_layered_image: Add docstrings to functions 2020-06-21 12:35:38 +12:00
Utku Demir
26402290bf
stream_layered_image: Add main method 2020-06-21 12:35:38 +12:00
Utku Demir
307804d97d
dockerTools.streamLayeredImage: comments 2020-06-21 12:35:38 +12:00
Utku Demir
4ab7baf6f6
stream_layered_image.py: comments 2020-06-21 12:35:38 +12:00
Utku Demir
a5a611cacb
Set mtime in even more places 2020-06-21 12:35:38 +12:00
Utku Demir
4e0109f873
Set mtime to get more deterministic builds 2020-06-21 12:35:37 +12:00
Utku Demir
560201da66
Implement dockerTools.streamLayeredImage 2020-06-21 12:35:32 +12:00