Commit graph

2348 commits

Author SHA1 Message Date
Naïm Favier
0b067316d4
stdenv: use intersectAttrs instead of intersectLists
Better complexity.
2022-12-17 12:55:44 +01:00
Naïm Favier
84eebc0fe4
Merge pull request #205944 from ncfavier/structured-attrs-env 2022-12-15 13:27:54 +01:00
Naïm Favier
e14de22618
stdenv: handle env gracefully
Derivations not using `__structuredAttrs` should not attempt to set
environment variables from `env`.

Derivations using `__structuredAttrs` should fail if `env` is not
exportable.
2022-12-15 13:27:11 +01:00
Artturin
630bb71ac5 stdenv: sort defaultNativeBuildInputs alphabetically 2022-12-13 01:12:20 +02:00
Artturin
b3717f6c14 stdenv: remove now unneeded linux conditional 2022-12-13 01:04:29 +02:00
Artturin
7866db71cc stdenv/generic: fix todo 2022-12-13 00:34:48 +02:00
Artturin
11c3127e38 stdenv: detect the type of variable in {prepend,append}ToVar
stdenv: error if using {prepend,append}ToVar on associative array

i don't know how to prepend to associative array
2022-12-10 04:42:36 +02:00
Artturin
89dc806f13 what to do about attrs.env or {} maybe have a empty env attrset always
so no need to use `or`
2022-12-08 21:11:35 +02:00
Artturin
02e3f51d27 darwin: use // for binutils-unwrapped and cctools to preserve the other
attributes
2022-12-08 21:09:02 +02:00
Artturin
bb914d8676 stdenv: export system pname name version for substituteAll
so we don't have to add these to the env attrset
2022-12-08 21:09:02 +02:00
Artturin
734d7df235 allow derivation attributes in env
derivations can be coerced to their output paths
2022-12-08 06:13:19 +02:00
Artturin
8ad0103a34 config.structuredAttrsByDefault: add option 2022-12-08 06:13:19 +02:00
Artturin
1c4820efdd work around a nix bug 2022-12-08 06:13:19 +02:00
Artturin
238a6053c4 stdenv: support opt-in __structuredAttrs
Co-authored-by: Robin Gloster <mail@glob.in>

stdenv: print message if structuredAttrs is enabled

stdenv: add _append

reduces the chance of a user doing it wrong

fix nix develop issue

output hooks don't work yet in nix develop though

making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.

lets instead make a function that gets the output names

reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.

use a function to get all output names instead of using $outputs

copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
2022-12-08 06:13:19 +02:00
Artturi
df3056bfac
Merge pull request #176597 from amjoseph-nixpkgs/pr/remove-redundant-file-buildInput
stdenv/cross: remove now-redundant `file` nativeBuildInput on mingw
2022-11-21 19:22:16 +02:00
github-actions[bot]
77fbd162df
Merge staging-next into staging 2022-11-20 18:01:45 +00:00
Artturin
341e6fd558 splice.nix: start deprecating nativeDrv and crossDrv 2022-11-19 00:04:54 +02:00
Robert Hensing
51518a5fd3 stdenv.tests: Add succeedOnFailure 2022-11-14 19:03:35 +01:00
Sandro Jäckel
9f4143e964
stdenv: fix succeedOnFailure 2022-11-12 19:25:14 +01:00
github-actions[bot]
f52955a521
Merge master into staging-next 2022-10-16 18:01:42 +00:00
piegames
87d738e864
Merge #195120: check-meta.nix: fix checkMetaRecursively option 2022-10-16 15:33:16 +02:00
github-actions[bot]
4f07e7f326
Merge master into staging-next 2022-10-15 06:07:54 +00:00
Robert Scott
692512da6a
Merge pull request #193415 from risicle/ris-nixpkgs-allow-nonsource-fix-1
stdenv/check-meta: fix support for `NIXPKGS_ALLOW_NONSOURCE=1`
2022-10-15 01:25:44 +01:00
arcnmx
466fd1439f check-meta.nix: fix checkMetaRecursively option
In specific cases, combining the `checkMeta` and `checkMetaRecursively`
config options would result in `error: infinite recursion encountered`

fixes #193296
2022-10-13 14:25:57 -07:00
Franz Pletz
3624ac2458
perl: fix build with libxcrypt 2022-10-09 18:07:54 +02:00
Martin Weinelt
253ca4957d Merge remote-tracking branch 'origin/master' into staging-next 2022-10-05 00:44:16 +02:00
Artturin
30cc0cde34 stdenv: complete the deprecation of stdenv.glibc 2022-10-03 20:19:54 +03:00
Robert Scott
d02ac63f4f stdenv/check-meta: fix support for NIXPKGS_ALLOW_NONSOURCE=1 2022-09-28 23:18:33 +01:00
github-actions[bot]
f18d801779
Merge staging-next into staging 2022-09-28 18:05:53 +00:00
aszlig
8f98a6d39b check-meta: Add isHydraChannel
This is needed in order to mark a certain derivation containing a Nix
expression tarball to Hydra so that it is recognised as a channel.

When I first got an evaluation error due to using this meta attribute, I
was under the impression that nobody outside of Vuizvui[1] is using this
feature and that we don't have any occurrence of isHydraChannel in
Nixpkgs.

However, when working around[2] the issue I assumed that it's not
something that should be included in Nixpkgs because we're not using it
there.

It turned out that my assumption was wrong and we *do* use the attribute
in Nixpkgs, namely via releaseTools.channel, which is similar to what
we're doing in Vuizvui.

Since we already include a bunch of undocumented attributes in
metaTypes, it only makes sense to add isHydraChannel as well since it's
actually documented in the Hydra documentation[3].

[1]: https://github.com/openlab-aux/vuizvui
[2]: https://github.com/openlab-aux/vuizvui/commit/e0685e81b3fdc43a272f0
[3]: 53335323ae/doc/manual/src/jobs.md (meta-fields)

Signed-off-by: aszlig <aszlig@nix.build>
2022-09-28 14:12:45 +02:00
github-actions[bot]
e0bea8e017
Merge staging-next into staging 2022-09-26 00:03:58 +00:00
piegames
6762de9a28 check-meta.nix: type checking changes
- Enable metadata checking by default, see https://github.com/NixOS/nixpkgs/pull/25304#issuecomment-298385426
- Check metadata before any other package issues, see https://github.com/NixOS/nixpkgs/issues/191124#issuecomment-1246523976
- Document that type checks only apply to the top level of nested values.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-09-25 16:37:15 +02:00
Graham Christensen
c2b898da76 treewide: drop -l$NIX_BUILD_CORES
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load.

For a build machine which is configured to run `$B` builds where each
build gets `total cores / B` cores (`$C`), passing `-l $C` to make will
improperly limit the load to `$C` instead of `$B * $C`.

This effect becomes quite pronounced on machines with 80 cores, with
40 simultaneous builds and a cores limit of 2. On a machine with this
configuration, Nix will run 40 builds and make will limit the overall
system load to approximately 2. A build machine with this many cores
can happily run with a load approaching 80.

A non-solution is to oversubscribe the machine, by picking a larger
`$C`. However, there is no way to divide the number of cores in a way
which fairly subdivides the available cores when `$B` is greater than
1.

There has been exploration of passing a jobserver in to the sandbox,
or sharing a jobserver between all the builds. This is one option, but
relatively complicated and only supports make. Lots of other software
uses its own implementation of `-j` and doesn't support either `-l` or
the Make jobserver.

For the case of an interactive user machine, the user should limit
overall system load using `$B`, `$C`, and optionally systemd's
cpu/network/io limiting features.

Making this change should significantly improve the utilization of our
build farm, and improve the throughput of Hydra.
2022-09-22 16:01:23 -04:00
Stéphan Kochen
6515b56871
darwin.apple_sdk_11_0: use stdenv objc4
Stdenv on aarch64-darwin pulls in (bootstrap-stage4) objc4, unlike
x86_64. However derivations that otherwise depend on objc4 would use a
a different objc4 derivation on top of the final stdenv.

Because this library defines an LLVM module, having multiple instances
of it in the import path will interfere with builds.
2022-09-20 12:14:49 +02:00
Vladimír Čunát
f08ac2c92b
Merge branch 'master' into staging-next 2022-08-29 08:58:52 +02:00
pennae
abd2e87da1 stdenv: remove deprecated adapters 2022-08-26 16:25:44 +02:00
Adam Joseph
51ba233717 pkgs/stdenv/linux: bootstrap-files for third (and final) mips ABI: n32
This PR provides the Hydra-generated bootstrap tarballs for mips64el-linux-gnuabin32.

With this PR we now have the bootstrap-file for all three little-endian ABIs on mips: n64, n32, and o32.  I do not currently plan to do big-endian mips unless some motivation arises; all mips chips are bi-endian and Debian has dropped big-endian support due to lack of interest.

I'll be following the script used in #151399, #168199, and #183487.

Files came from [this](https://hydra.nixos.org/build/188389586#tabs-summary) Hydra build, which used nixpkgs revision 97d9c84e1d to instantiate:

```
/nix/store/hakn8s85s9011v61r6svp5qy8x1y64fv-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32.drv
```

and then built:

```
/nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32
```

I downloaded these files from Hydra and prefetched them into the nix store with the following commands:

```
STOREPATH=rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store prefetch-file \
  file://$(nix store add-file --name bootstrap-tools.tar.xz  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix store prefetch-file --executable \
  file://$(nix store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
```

These commands produced the following output:

```
Downloaded 'file:///nix/store/w6zzd2fx2vhmjfcf5h5zc01m0swldpbw-bootstrap-tools.tar.xz' to '/nix/store/6w0f0mqblrghvh6yjwcb4xdqq9x50lbl-w6zzd2fx2vhmjfcf5h5zc01m0swldpbw-bootstrap-tools.tar.xz' (hash 'sha256-LWrpN6su2yNVurUyhZP34OiZyzgh7MfN13fIIbou8KI=').
Downloaded 'file:///nix/store/nqagw1kgdz1zlmqi00qfjrmwqk3g3bgd-busybox' to '/nix/store/i361xhbdhhnvg7zd637xpm63vbl80s0s-nqagw1kgdz1zlmqi00qfjrmwqk3g3bgd-busybox' (hash 'sha256-4N3G1qYA7vitjhsIW17pR6UixIuzrq4vZXa8F0/X4iI=').
```

I used the hashes from the output above to create the `fetchurl` invocation which is part of this commit.

I then started the bootstrap with the following command:

```
nix build -L -f . --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabin32' hello
```

As @lovesegfault requested, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
sha256sum /nix/store/${STOREPATH}/on-server/*
```

which produced the following output:

```
2d6ae937ab2edb2355bab5328593f7e0e899cb3821ecc7cdd777c821ba2ef0a2  /nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32/on-server/bootstrap-tools.tar.xz
01633f71135cb9ab1b5ce3ebb67e80cbf288739729bffc1350c1552f6f8df34b  /nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32/on-server/busybox
```
2022-08-25 16:13:37 -07:00
github-actions[bot]
0b8aa77df1
Merge master into staging-next 2022-08-25 18:01:10 +00:00
Bernardo Meurer
14181ab152
Merge pull request #185182 from amjoseph-nixpkgs/pr/powerpc64le-bootstrapFiles/again
pkgs/stdenv/linux: update powerpc64le bootstrap-files
2022-08-25 10:07:22 -03:00
Matthew Bauer
ec4019f6dc Set inherit_errexit after bash version check
inherit_errexit wasn’t available in bash 3. We have a check to show a
nice error message, but that check is after we set inherit_errexit in
setup.sh. So we can just move this to below the BASH_VERSINFO check.
2022-08-21 23:01:55 -04:00
Sergei Trofimovich
a0219e7fb7
Merge pull request #181536 from trofi/fix-gnumake-guile
gnumake: unconditionally disable guileSupport on bootstrap
2022-08-19 19:22:13 +01:00
Sergei Trofimovich
7f92320f66 make-bootstrap-tools.nix: drop libelf.so from tootstrap tarballs
gcc stopped using libelf in commit 48215350c24 ("re PR lto/46273 (Failed
to bootstrap)") around 2010, before gcc-4.6.0.

Bootstrap tools don't use it either.
2022-08-18 08:23:08 +01:00
Bernardo Meurer
d00f5c5eaa
Merge pull request #183487 from amjoseph-nixpkgs/pr/mipsel/bootstrap-files
stdenv/linux/bootstrap-files: add mipsel-linux (mips32)
2022-08-17 15:14:47 -07:00
Adam Joseph
6122297eab pkgs/stdenv/linux: update powerpc64le bootstrap-files
This PR updates the Hydra-generated bootstrap tarballs for
powerpc64le-linux.  The bootstrap-files referenced prior to this
commit will only bootstrap in a nixpkgs which has
https://github.com/NixOS/nixpkgs/pull/181802.  That PR was closed in
favor of https://github.com/NixOS/nixpkgs/pull/181943, which is a
cleaner solution but which requires regenerating the bootstrap-files.

I'll be following the script established in
https://github.com/NixOS/nixpkgs/pull/151399, which I previously used
in https://github.com/NixOS/nixpkgs/pull/168199.

Files came from [this](https://hydra.nixos.org/build/186237511) Hydra build, which used nixpkgs revision ac43c44478 to instantiate:

```
/nix/store/nhjbza9vlcyhp9zxfz6lwpc3m2ghrpzj-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu.drv
```

and then built:

```
/nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
```

I downloaded these files from Hydra with the following commands:

```
STOREPATH=fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

I then prefetched them into `/nix/store` with:

```
$ nix store prefetch-file --executable file:///nix/store/p9lz8r81zp3a4sl2qq2v4j69syjzryn2-busybox
Downloaded 'file:///nix/store/p9lz8r81zp3a4sl2qq2v4j69syjzryn2-busybox' to '/nix/store/a42qf2kf5hychcsw5sz0pvghy9vli1im-p9lz8r81zp3a4sl2qq2v4j69syjzryn2-busybox' (hash 'sha256-jtPEAsht4AUAG4MLK8xocQSfveUR4ppU1lS4bGI1VN4=').

$ nix store prefetch-file file:///nix/store/y4530zpk7ia4szf5cdi4zpyy5lpjv3iv-bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/y4530zpk7ia4szf5cdi4zpyy5lpjv3iv-bootstrap-tools.tar.xz' to '/nix/store/kgzyq9q08nll28ccqjcbv8angq5hyvdp-y4530zpk7ia4szf5cdi4zpyy5lpjv3iv-bootstrap-tools.tar.xz' (hash 'sha256-MpIDnpZUK3M17qlnuoxfnK0EgxRosm3TMW1WfPZ1+jU=').
```

And started the bootstrap with the following command:

```
nix build -f . -L hello
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/*
3292039e96542b7335eea967ba8c5f9cad04831468b26dd3316d567cf675fa35  /nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
3d078dff7b4087d82442937667c91dace3321493aae4d3a4160d046b7eabcc2c  /nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/busybox
```
2022-08-04 23:05:31 -07:00
github-actions[bot]
2a54df0946
Merge staging-next into staging 2022-08-03 18:02:18 +00:00
Sandro
0e0cc47192
Merge pull request #181550 from reckenrode/update-darwin-bootstrap-tools 2022-08-03 16:12:31 +02:00
Sandro
09cb5e8416
Merge pull request #183507 from Artturin/stdenvmeson1 2022-08-01 23:06:51 +02:00
github-actions[bot]
e8ce2f4a26
Merge master into staging-next 2022-08-01 18:01:20 +00:00
Artturin
9f309c8792 stdenv: mesonFlags: use canExecute in needs_exe_wrapper
this is more accurate
2022-08-01 20:03:47 +03:00
Robert Hensing
12a060dbbc
Merge pull request #183967 from Ericson2314/small-stdenv-cleanup
stdenv: Two small cleanups
2022-08-01 14:37:42 +02:00
github-actions[bot]
15686bdd94
Merge master into staging-next 2022-07-30 18:01:31 +00:00
Alyssa Ross
5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
John Ericson
650380ca0b stdenv: Simplify dispatch
This is a change for `powerpc-linux` but that is ancient and I don't
think it matters. The impure bootstrap that was previously assigned to
it has probably bitrotted anyways.
2022-07-29 18:34:02 -04:00
John Ericson
9d59a4df78 stdenv/common-path.nix: move to stdenv/generic/
Do this since it is part of the generic stdenv/`mkDerivation`
infrastructure, rather than being a bootstrapping strategy.
2022-07-29 18:24:00 -04:00
Sergei Trofimovich
f6ee60dde4 gnumake: unconditionally disable guileSupport on bootstrap
Before the change an attempt to use `gnumake.override { guileSupport = true; }`
caused recursion in bootstrap stages as guileSupport pulls in guile and it's
dependencies.

To restore the bootstrap the change unconditionally sets
`guileSupport = false;` for `gnumake`.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-07-29 19:34:09 +01:00
Adam Joseph
efb35dff8a stdenv/linux/bootstrap-files: add mipsel-linux (mips32) 2022-07-28 14:23:11 -07:00
github-actions[bot]
9153131664
Merge staging-next into staging 2022-07-27 12:02:00 +00:00
github-actions[bot]
9ed58a6fb6
Merge master into staging-next 2022-07-27 12:01:18 +00:00
Vladimír Čunát
a0e2bdeb84
Merge #179628: make-bootstrap-tools-cross: add mips32el 2022-07-27 13:12:10 +02:00
Artturi
79e8669003
Merge pull request #179603 from Artturin/subplacemultiple 2022-07-24 04:14:09 +03:00
Artturi
111abd87ca
Merge pull request #168590 from amjoseph-nixpkgs/stdenv-disallowedReferences
stdenv: use disallowedRequisites to check forbidden requisites
2022-07-18 02:51:23 +03:00
github-actions[bot]
8df1eb061a
Merge master into staging-next 2022-07-17 00:02:14 +00:00
Bernardo Meurer
5fed5fe917
Merge pull request #180223 from amjoseph-nixpkgs/pr/mips64el/bootstrap-file-hashes
pkgs/stdenv/linux: add mips64el bootstrap-files
2022-07-16 16:41:08 -07:00
Randy Eckenrode
8d1c1184e6
make-bootstrap-tools: add pbzx and tbd tools on x86_64-darwin 2022-07-14 21:07:28 -04:00
Artturin
ca2ca8b194 stdenv: start deprecating non-list mesonFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that mesonFlags should be a list of strings
2022-07-06 15:23:20 +03:00
Adam Joseph
73811b372e pkgs/stdenv/linux: add mips64el bootstrap-files
This PR adds Hydra-generated bootstrap tarballs for mips64el-linux.  I'll be following the script established in https://github.com/NixOS/nixpkgs/pull/151399, which I previously used in https://github.com/NixOS/nixpkgs/pull/168199.

Files came from [this](https://hydra.nixos.org/build/182757245) Hydra build, which used nixpkgs revision ef3fe254f3 to instantiate:

```
/nix/store/a2bvv663wjnyhq8m7v84aspsd3sgf9h6-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64.drv
```

and then built:

```
/nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
```

I downloaded these files from Hydra with the following commands:

```
STOREPATH=aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

I then prefetched them into `/nix/store` with:

```
$ nix store prefetch-file --executable file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox' to '/nix/store/ai30ss23914syz6j8m95arkwffbbx44k-busybox' (hash 'sha256-sTE58ofjqAqX3Xtq1g9wDxzIe6Vo//GHbicfqJoivDI=').

$ nix store prefetch-file  file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz' to '/nix/store/nr6zjrbwbxcxs6brf042zsyqllwbwj9v-bootstrap-tools.tar.xz' (hash 'sha256-tTgjeXpd2YgnfP4JvRuO0bXd2j8GqzBcd57JI3wH9x0=').
```

And started the bootstrap with the following command (the `--arg localSystem` is needed because #161159 has not merged):

```
nix build -f . -L hello --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabi64'
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/*
b53823797a5dd988277cfe09bd1b8ed1b5ddda3f06ab305c779ec9237c07f71d  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
598e05abb69b2c1a0db46585cd2131212077c0937ce2a665daf3811f059ae767  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
```
2022-07-05 12:16:58 -07:00
Artturin
3173c3b6b6 stdenv: start deprecating non-list cmakeFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that cmakeFlags should be a list of strings
2022-07-05 04:05:30 +03:00
Adam Joseph
dc5bdd6852 Revert "stdenv: label the ephemeral coreutils-stage4 package"
This reverts commit 23ea8b35da.
2022-07-02 16:56:08 -07:00
Rick van Schijndel
2fcdf54568
Merge pull request #169378 from amjoseph-nixpkgs/stdenv-makeStaticLibraries-do-what-comment-says
stdenv: make stage3.{gmp,mpfr,mpc,isl} do what the comment says
2022-06-30 07:44:05 +02:00
Adam Joseph
ea32f3e523 make-bootstrap-tools-cross: enable Hydra builds of mips32el bootstrap-files
This one-line commit will cause Hydra to build a bootstrap-files
tarball for 32-bit little-endian MIPS.
2022-06-29 14:48:13 -07:00
Artturin
ba1efa71ae stdenv: substituteInPlace: accept multiple filenames
I don't know if getopt is available everywhere, so I did not use it.

in any case, it can be changed to use getopt in the future if
needed.
2022-06-29 22:03:19 +03:00
github-actions[bot]
5b9ee17283
Merge master into staging-next 2022-06-24 00:02:29 +00:00
Sergei Trofimovich
b0f5f5cc96
Merge pull request #178408 from trofi/add-config.enableParallelBuildingByDefault
config.configurePlatformsByDefault: init option
2022-06-23 21:32:59 +00:00
Rick van Schijndel
adafa1cb42
Merge pull request #178626 from a-m-joseph/pr/168413fallout/mktemp
setup.sh: use six `X` characters in `mktemp` invocation
2022-06-23 06:45:40 +02:00
Rick van Schijndel
f104ffc941
Merge pull request #177789 from a-m-joseph/pr/setup/busyboxfind
setup.sh: use -exec rather than -execdir
2022-06-23 06:40:38 +02:00
Adam Joseph
97bf76b3b3 setup.sh: use six X characters in mktemp invocation
Closes #178625

The `busybox` version of `mktemp` requires exactly six `X` characters
in the argument to `mktemp`, unlike the `coreutils` version of `mktemp`.

Let's accomodate packages, like `epson-escpr2`, which fool `setup.sh`
into using the `busybox` version instead of the `stdenv` version.
2022-06-22 12:46:22 -07:00
github-actions[bot]
30c9243acc
Merge staging-next into staging 2022-06-22 00:03:15 +00:00
Sergei Trofimovich
c534d056fe pkgs/stdenv/generic/make-derivation.nix: add a bug reference to strictDepsByDefault TODO 2022-06-21 19:03:43 +01:00
Sergei Trofimovich
db09d70d33 config.configurePlatformsByDefault: init option
Useful to enable tree-wide occasionally to have incremental progress
towards https://github.com/NixOS/nixpkgs/pull/87909 resolution.
2022-06-21 09:23:13 +01:00
Vladimír Čunát
5ffd19ddbf
Merge #175785: libidn2: hack to avoid referencing bootstrap tools
...into staging
2022-06-17 18:23:25 +02:00
Adam Joseph
64fc13ff1d setup.sh: use -exec rather than -execdir
Busybox's `find` does not support `-execdir`, so let's uses `-exec`
instead, in the spirit of requiring the minimum features needed for
the task.
2022-06-15 12:10:44 -07:00
github-actions[bot]
caada97a27
Merge master into staging-next 2022-06-14 00:02:11 +00:00
Robert Schütz
38c776b679 stdenv/check-meta: support NIXPKGS_ALLOW_NONSOURCE=0 2022-06-13 19:08:17 +00:00
Martin Weinelt
6451e9d4a5 Merge remote-tracking branch 'origin/staging-next' into staging 2022-06-08 22:22:26 +02:00
Alyssa Ross
5dfe7f928e
release-cross.nix: fix cross bootstrap tools eval
In 5643714dea, I changed pkgs/stdenv/linux/make-bootstrap-tools.nix
to take a package set instead of system and localSystem arguments, but
I forgot to update make-bootstrap-tools-cross.nix.

Fixes: 5643714dea ("stdenvBootstrapTools: inherit {cross,local}System")
2022-06-08 17:17:10 +00:00
Adam Joseph
10030672ab stdenv/cross: remove now-redundant file nativeBuildInput on mingw
Since 97c43828fb the `file` package has
been part of stdenv, and no longer needs to be listed explicitly as a
build input.  Let's remove the platform-specific inclusion for mingw64
as suggested by @mehmooda:

  https://github.com/NixOS/nixpkgs/pull/168413#issuecomment-1147370500

I traced the line removed by this commit through the `git blame`; it
was initially added in this commit (and then shuffled around a few
dozen times by refactorings):

  8b292a1b35

The commit message indicates that `libpng-1.6.20` was current at the
time.  Although there are [libpng
archives](https://github.com/glennrp/libpng) available in git form,
the older versions don't have their autoconfery vendored in, so I
can't link to them.  Fortunately the relevant bit hasn't changed since
then:

a37d483651/configure (L5575)

```
mingw* | pw32*)
  # Base MSYS/MinGW do not provide the 'file' command needed by
  # func_win32_libid shell function, so use a weaker test based on 'objdump',
  # unless we find 'file', for example because we are cross-compiling.
  if ( file / ) >/dev/null 2>&1; then
    lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL'
    lt_cv_file_magic_cmd='func_win32_libid'
  else
    # Keep this pattern in sync with the one in func_win32_libid.
    lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)'
    lt_cv_file_magic_cmd='$OBJDUMP -f'
  fi
  ;;
```
2022-06-06 11:26:46 -07:00
github-actions[bot]
a1bbd60cb7
Merge staging-next into staging 2022-06-05 18:01:45 +00:00
John Ericson
82707a2593
Merge pull request #176385 from hercules-ci/fix-overlay-style-stdenv-adapters
stdenv/adapters.nix: Fix for overlay style arguments
2022-06-05 12:29:24 -04:00
Robert Hensing
dd770cc211 stdenv/adapters.nix: Fix for overlay style arguments 2022-06-05 13:54:03 +02:00
Robert Hensing
cd88f8613f pkgs/make-derivation.nix: Refactor, hardcode mkDerivationSimple 2022-06-05 13:36:56 +02:00
Robert Hensing
7e3c80f5b7 pkgs/make-derivation.nix: Refactor, inline makeOverlayable 2022-06-05 13:35:04 +02:00
Robert Hensing
1b00adbad1 pkgs/make-derivation.nix: Refactor, introduce let binding 2022-06-05 13:33:35 +02:00
Adam Joseph
02630180fa stdenv: add -stageX markers to gmp, mpfr, libmpc, and isl 2022-06-05 00:35:06 -07:00
Adam Joseph
a9e0d86411 Update pkgs/stdenv/linux/default.nix
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
2022-06-05 00:35:06 -07:00
Adam Joseph
23ea8b35da stdenv: label the ephemeral coreutils-stage4 package
During stdenv bootstrapping, coreutils is built twice.  This makes
troubleshooting very difficult, because both packages have
name="coreutils", so it is a hassle to figure out "which coreutils am
I using / is not building"?

The first of these builds is used only in stage4, and is not part of
the final stdenv.  Let's label that one with a different `name`
attribute to make it obvious which is which.
2022-06-05 00:35:06 -07:00
Adam Joseph
122b6930b0 stdenv: cause makeStaticLibraries usage to agree with usage spec
The usage of `makeStaticLibraries` in stdenv/linux/default.nix is
prefaced by this comment:

  # Link GCC statically against GMP etc.  This makes sense because
  # these builds of the libraries are only used by GCC, so it
  # reduces the size of the stdenv closure.

However "these builds of the libraries are only used by GCC" is not
actually true.  As currently written, the stage4 coreutils links
against these customized, static-ified libraries.

Beside the fact that the code doesn't actually do what it says, this
causes other problems as well.  One example is #168983, which arises
because have a dynamically-linked binary (coreutils) which is built
from statically-linked libraries (libgmp.a); doing this causes mayhem
on platforms where `-fstack-protector` needs an auxiliary
`libssp.{so,a}` library; we end up with link failures because some
parts of the resulting binary want `libssp.so` and other parts want
`libssp_nonshared.a`.

Let's make the code actually do what the comment says, by moving these
definitions into the `gcc-unwrapped` override.  This will cause the
stage4-coreutils to link against libgmp dynamically, rather than
statically.  For this reason this commit depends on the previous
commit, which allows that to be done without creating a forbidden
reference from stdenv-final to the bootstrap-files.
2022-06-05 00:35:06 -07:00
Adam Joseph
7fd749009f stdenv: force gmp to rebuild in stage4 of the bootstrap
As explained in the comment, this ensures that stage4-coreutils does
not leak a reference to the bootstrap-files by way of libgmp.  This
will allow the next patch in this series to build stage4-coreutils
using a dynamically-linked (rather than statically-linked) libgmp.
2022-06-05 00:35:00 -07:00
Vladimír Čunát
088b29159d
libidn2: hack to avoid referencing bootstrap tools
Due to bootstrap tools getting purged from closure of libidn2.dev,
a very large rebuild is caused.
2022-06-05 08:59:20 +02:00
github-actions[bot]
dc4d841894
Merge staging-next into staging 2022-06-02 00:03:03 +00:00
github-actions[bot]
d6b9d24302
Merge master into staging-next 2022-06-02 00:02:21 +00:00
Sandro
4c3e9f091a
check-meta: fix comment (#175517)
* check-meta: fix comment

* Apply suggestions from code review
2022-06-02 01:38:57 +02:00
github-actions[bot]
e8d5ce419a
Merge staging-next into staging 2022-05-31 18:01:50 +00:00
github-actions[bot]
d6ede66f6a
Merge master into staging-next 2022-05-31 18:01:11 +00:00
Alyssa Ross
5643714dea stdenvBootstrapTools: inherit {cross,local}System
It's expected that attributes in the top-level package set will all use
that package set, but this wasn't the case for the bootstrap tools.
This led some very confusing behaviour:

- pkgsMusl.stdenvBootstrapTools would build glibc bootstrap tools
- stdenvBootstrapTools was _always_ cross compiled, even if
  Nixpkgs wasn't, because it always set crossSystem.  This also didn't
  match the behaviour of using make-bootstrap-tools.nix as an
  entrypoint, where crossSystem would default to null.

For the Linux stdenv, I've made the ideal fix, which is to make pkgs an
argument rather than taking the arguments for pkgs, and then
re-importing it.  This means it'll always use exactly the same package
set that's calling it, and should also mean faster eval due to not
importing Nixpkgs twice.

The Darwin stdenv is more complicated, and I'm not able to easily test
it, so I wasn't confident in making the same fix there.  Instead, I've
just made sure crossSystem and localSystem are set to the correct values
so they're not always cross compiled and match the parent package set's.
It would still be preferable if somebody could make Darwin's
make-bootstrap-tools.nix take pkgs as an argument, rather than all the
arguments for pkgs.
2022-05-31 14:32:27 +00:00
github-actions[bot]
2d502cf3a4
Merge staging-next into staging 2022-05-30 12:02:11 +00:00
github-actions[bot]
76b451751c
Merge master into staging-next 2022-05-30 12:01:31 +00:00
Robert Scott
5bb9bf4774 meta.sourceProvenance: inline hasSourceProvenance
it may be what the license handling code does, but it's confusing and not very useful

Co-authored-by: Adam Joseph <54836058+a-m-joseph@users.noreply.github.com>
2022-05-30 16:27:34 +08:00
Robert Scott
7906ea6d9d allowNonSourcePredicate: use example of categorical permissivity
Co-authored-by: Adam Joseph <54836058+a-m-joseph@users.noreply.github.com>
2022-05-30 16:27:34 +08:00
Robert Scott
095eb91533 meta.sourceProvenance: disallow string values
strings complicate reasoning about values and may not be needed with `sourceProvenance`

Co-authored-by: Alexander Foremny <aforemny@posteo.de>
2022-05-30 16:27:34 +08:00
Robert Scott
da9162f667 add mechanism for handling meta.sourceProvenance attributes
heavily based on patterns used by licenses infrastructure, so may
appear overengineered for its initial level of use
2022-05-30 16:27:34 +08:00
Rick van Schijndel
f174277eed
Merge pull request #168413 from a-m-joseph/libtool-purity-fix-put-file-in-bootstrap
fixLibtool(): replace /usr/bin/file in ./configure, add file to common-path.nix
2022-05-28 17:53:43 +02:00
github-actions[bot]
ff43d848bf
Merge staging-next into staging 2022-05-27 18:01:52 +00:00
Artturi
60a0871486
Merge pull request #172391 from Artturin/deprecate-stdenv-glibc 2022-05-27 17:56:19 +03:00
Alyssa Ross
089ff89f49
lib.systems: drop scaleway-c1
These servers apparently no longer exist, since September 2, 2021[1].
If somebody needs this for non-Scaleway machines, they should suggest
its reintroduction with a different name.

[1]: https://news.ycombinator.com/item?id=27192757
2022-05-27 12:02:39 +00:00
Adam Joseph
97c43828fb fixLibtool(): patch ./configure, add file to common-path.nix
libtool's libtool.m4 script assumes that `file` is available, and can
be found at `/usr/bin/file` (this path is hardwired).  Furthermore,
the script with this assumption is vendored into the ./configure
scripts of an enormous number of packages.  Without this commit, you
will frequently see errors like this during the configurePhase with
the sandbox enabled:

  ./configure: line 9595: /usr/bin/file: command not found

Due mostly to luck, this error does not affect native compiles on
nixpkgs' two most popular platforms, x86_64-linux and aarch64-linux.
However it will cause incorrect linker flag detection and a failure to
generate shared libraries for sandboxed cross-builds to a x86_64-linux
host as well as any sandboxed build (cross or native) for the following
hosts: x86_64-freebsd, *-hpux, *-irix, mips64*-linux, powerpc*-linux,
s390x-linux, s390x-tpf, sparc-linux, and *-solaris.

This commit fixes the problem by adding an extra line to fixLibtool()
in pkgs/stdenv/generic/setup.sh.  This extra line will scan the
unpacked source code for executable files named "configure" which
contain the following text:

'GNU Libtool is free software; you can redistribute it and/or modify'

This text is taken to be an indicator of a vendored libtool.m4.  When
it is found, the configure script containing it is subjected to `sed
-i s_/usr/bin/file_file_` which replaces all occurrences of
`/usr/bin/file` with `file`.

Additionally, the `file` package is now considered to be part of
`stdenv`.  It has been added to `common-path.nix` so that the `file`
binary will be found in the `$PATH` of every build, except for the
bootstrap-tools and the first few stages of stdenv boostrapping.

Verified no regressions under:

  nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix

This commit allows the following commands to complete, which should
enable Hydra to produce bootstrap-files for mips64el:

  nix-build \
    --option sandbox true \
    --option sandbox-fallback false \
    pkgs/top-level/release-cross.nix \
    -A bootstrapTools.mips64el-linux-gnuabi64.build

  nix-build \
    --option sandbox true \
    --option sandbox-fallback false \
    . \
    -A pkgsCross.mips64el-linux-gnuabi64.nix_2_4
2022-05-26 13:53:36 -07:00
Artturin
1d44ac176c treewide: add enableParallelBuilding's to bootstrap packages so hashes stay the same
when enableParallelBuildingByDefault is enabled

verified with
`nix-diff $(nix eval ".#gcc-unwrapped.drvPath") $(nix eval --expr 'with import ./. { config = { enableParallelBuildingByDefault = true; }; }; gcc-unwrapped.drvPath' --impure)`
2022-05-25 16:03:14 +03:00
Artturin
0c4d65b21e treewide: stdenv.glibc -> glibc 2022-05-25 15:51:20 +03:00
Artturin
a05b581783 stdenv: warn about use of stdenv.glibc
TODO was added in
119920faa6
2022-05-25 15:51:20 +03:00
github-actions[bot]
574f2d4696
Merge staging-next into staging 2022-05-24 00:02:57 +00:00
sternenseemann
41485e7337 stdenv.mkDerivation: be less strict about check execution for cross
Instead of requiring the platforms be equal, use `isCompatible` to
determine if we can execute tests. The upside of this is that we now
can execute tests for natively cross compiled package sets like
pkgsStatic, pkgsLLVM and pkgsCross.musl64 etc.
2022-05-23 21:25:04 +02:00
sternenseemann
acb063701a lib.systems.elaborate: expose canExecute predicate over isCompatible
canExecute is like isCompatible, but also checks that the Kernels are
_equal_, i.e. that both platforms use the same syscall interface. This
is crucial in order to actually be able to execute binaries for the
other platform.

isCompatible is dropped, since it has changed semantically and there's
no use case left in nixpkgs.
2022-05-23 21:25:04 +02:00
Artturin
f002ffed9a treewide: enable strictDeps in bootstrap packages 2022-05-22 16:40:26 +03:00
Artturin
6d54fe622d stdenv: start deprecating non-list configureFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that configureFlags should be a list of strings
2022-05-18 20:32:33 +03:00
Artturi
22f6a17b22
Merge pull request #172570 from Artturin/epbbydefault 2022-05-13 20:46:33 +03:00
Artturin
a17e1b8809 config.enableParallelBuildingByDefault: init option
useful for testing if some builds could be faster
2022-05-11 16:41:30 +03:00
Linus Heckemann
676e4d42d6
Merge pull request #157106 from lheckemann/better-error-for-bad-buildinputs
stdenv/make-derivation: check that all inputs are of an appropriate type
2022-05-11 11:28:11 +02:00
github-actions[bot]
f39e774f11
Merge master into staging-next 2022-05-09 00:02:27 +00:00
John Ericson
db0c4f14ac
Merge pull request #172022 from elikoga/patch-2
Update link from old nix manual to current stable one
2022-05-08 18:26:19 -04:00
elikoga
04f6a432c4
Update link from old nix manual to current stable one
https://github.com/NixOS/rfcs/pull/72
2022-05-08 05:20:34 +02:00
Artturin
2d019e0931 stdenv: deprecate unused adapters
stdenv: deprecate addCoverageInstrumentation adapter

this used to be used in nixos/tests but it hasn't been used in nixpkgs
for years

stdenv: deprecate replaceMaintainersField adapter

it was added in 2009 in 01e98e49b1
by nbp
there are no uses of it in nixpkgs now

stdenv: deprecate validateLicenses adapter

it was added in 2009 in b29073af25

unfreePredicate is now handled in ./pkgs/stdenv/generic/check-meta.nix
so this is unnecessary
2022-05-07 22:36:10 +03:00
Linus Heckemann
b3e8855999 make-derivation: allow strings in build input lists 2022-05-07 10:42:50 +02:00
github-actions[bot]
16684f8bd3
Merge master into staging-next 2022-05-04 12:01:10 +00:00
Artturi
f0e48dd98b
Merge pull request #119942 from hercules-ci/mkDerivation-overridable-recursive-attributes 2022-05-04 14:31:50 +03:00
github-actions[bot]
ee7e3f30f3
Merge staging-next into staging 2022-05-04 00:02:57 +00:00
ckie
4def222ea4
stdenv/check-meta: add a "maintainerless" warning
This warning logs when a package has no maintainers. It will stay silent
if `meta.maintainers` is not set at all, only complaining when it is an
empty list. In the future a separate warning could be added to allow for
that stricter behavior. Or this warning could be changed.
2022-05-03 22:29:14 +03:00
ckie
3a34b6c820
stdenv/check-meta: add an eval warning option
This will be used in the next commit in this patch series.
2022-05-03 22:29:12 +03:00
ckie
5e420c2455
stdenv/check-meta: turn validity.valid into a str
This will allow for adding more validity types in the future, such as a
warning type. (which is in the next commit in this series)

This is NOT a breaking change because validity.valid is never exposed
outside of `stdenv.mkDerivation`.
2022-05-03 22:28:25 +03:00
github-actions[bot]
51a899bdc1
Merge staging-next into staging 2022-05-03 18:10:46 +00:00
Artturin
9f05fc6661 config.allowUnsupportedSystem: define as option 2022-05-02 20:39:43 +03:00
Artturin
9f473092f8 config.allowBroken: define as option 2022-05-02 17:20:44 +03:00
Artturin
1c49b81263 config.allowUnfree: define as option 2022-05-02 17:20:25 +03:00
Robert Hensing
0e00acafe9 stdenv.mkDerivation: public -> finalPackage 2022-05-02 08:49:33 +02:00
Robert Hensing
37ab5b4396 mkDerivation: Add error hint for infinite recursion 2022-05-02 08:49:32 +02:00
Robert Hensing
41b3688ba1 make-derivation.nix: Remove unnecessary TODO
The goal was to keep the number of calls to a minimum, but that
would come at the cost of clobbering git blame and possibly not
improving readability.
2022-05-02 08:49:31 +02:00
Robert Hensing
6d7efb3a16 stdenv.mkDerivation: Make self more overlay-like; use self.public
`self` is now arguments, like `super`. The final package is in
`self.public`.
2022-05-02 08:49:31 +02:00
Robert Hensing
a4e7085227 stdenv.mkDerivation: Allow overriding of recursive definitions
See updated manual for further explanation.
2022-05-02 08:49:26 +02:00
Ben Siraphob
02dbca8c2b
Merge pull request #161270 from Stunkymonkey/darwin-phases 2022-04-30 10:58:05 -06:00
Artturin
33415135b0 config.contentAddressedByDefault: init option 2022-04-27 23:21:32 +03:00
Artturin
d76021402d config.strictDepsByDefault: init option
so i won't have to change make-derivation.nix when i wan't to make this
true
2022-04-27 23:21:21 +03:00
github-actions[bot]
deb60260a7
Merge master into staging-next 2022-04-17 06:01:07 +00:00