Commit graph

258 commits

Author SHA1 Message Date
Stéphan Kochen
6515b56871
darwin.apple_sdk_11_0: use stdenv objc4
Stdenv on aarch64-darwin pulls in (bootstrap-stage4) objc4, unlike
x86_64. However derivations that otherwise depend on objc4 would use a
a different objc4 derivation on top of the final stdenv.

Because this library defines an LLVM module, having multiple instances
of it in the import path will interfere with builds.
2022-09-20 12:14:49 +02:00
Sandro
0e0cc47192
Merge pull request #181550 from reckenrode/update-darwin-bootstrap-tools 2022-08-03 16:12:31 +02:00
John Ericson
9d59a4df78 stdenv/common-path.nix: move to stdenv/generic/
Do this since it is part of the generic stdenv/`mkDerivation`
infrastructure, rather than being a bootstrapping strategy.
2022-07-29 18:24:00 -04:00
Randy Eckenrode
8d1c1184e6
make-bootstrap-tools: add pbzx and tbd tools on x86_64-darwin 2022-07-14 21:07:28 -04:00
github-actions[bot]
e8d5ce419a
Merge staging-next into staging 2022-05-31 18:01:50 +00:00
Alyssa Ross
5643714dea stdenvBootstrapTools: inherit {cross,local}System
It's expected that attributes in the top-level package set will all use
that package set, but this wasn't the case for the bootstrap tools.
This led some very confusing behaviour:

- pkgsMusl.stdenvBootstrapTools would build glibc bootstrap tools
- stdenvBootstrapTools was _always_ cross compiled, even if
  Nixpkgs wasn't, because it always set crossSystem.  This also didn't
  match the behaviour of using make-bootstrap-tools.nix as an
  entrypoint, where crossSystem would default to null.

For the Linux stdenv, I've made the ideal fix, which is to make pkgs an
argument rather than taking the arguments for pkgs, and then
re-importing it.  This means it'll always use exactly the same package
set that's calling it, and should also mean faster eval due to not
importing Nixpkgs twice.

The Darwin stdenv is more complicated, and I'm not able to easily test
it, so I wasn't confident in making the same fix there.  Instead, I've
just made sure crossSystem and localSystem are set to the correct values
so they're not always cross compiled and match the parent package set's.
It would still be preferable if somebody could make Darwin's
make-bootstrap-tools.nix take pkgs as an argument, rather than all the
arguments for pkgs.
2022-05-31 14:32:27 +00:00
Adam Joseph
97c43828fb fixLibtool(): patch ./configure, add file to common-path.nix
libtool's libtool.m4 script assumes that `file` is available, and can
be found at `/usr/bin/file` (this path is hardwired).  Furthermore,
the script with this assumption is vendored into the ./configure
scripts of an enormous number of packages.  Without this commit, you
will frequently see errors like this during the configurePhase with
the sandbox enabled:

  ./configure: line 9595: /usr/bin/file: command not found

Due mostly to luck, this error does not affect native compiles on
nixpkgs' two most popular platforms, x86_64-linux and aarch64-linux.
However it will cause incorrect linker flag detection and a failure to
generate shared libraries for sandboxed cross-builds to a x86_64-linux
host as well as any sandboxed build (cross or native) for the following
hosts: x86_64-freebsd, *-hpux, *-irix, mips64*-linux, powerpc*-linux,
s390x-linux, s390x-tpf, sparc-linux, and *-solaris.

This commit fixes the problem by adding an extra line to fixLibtool()
in pkgs/stdenv/generic/setup.sh.  This extra line will scan the
unpacked source code for executable files named "configure" which
contain the following text:

'GNU Libtool is free software; you can redistribute it and/or modify'

This text is taken to be an indicator of a vendored libtool.m4.  When
it is found, the configure script containing it is subjected to `sed
-i s_/usr/bin/file_file_` which replaces all occurrences of
`/usr/bin/file` with `file`.

Additionally, the `file` package is now considered to be part of
`stdenv`.  It has been added to `common-path.nix` so that the `file`
binary will be found in the `$PATH` of every build, except for the
bootstrap-tools and the first few stages of stdenv boostrapping.

Verified no regressions under:

  nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix

This commit allows the following commands to complete, which should
enable Hydra to produce bootstrap-files for mips64el:

  nix-build \
    --option sandbox true \
    --option sandbox-fallback false \
    pkgs/top-level/release-cross.nix \
    -A bootstrapTools.mips64el-linux-gnuabi64.build

  nix-build \
    --option sandbox true \
    --option sandbox-fallback false \
    . \
    -A pkgsCross.mips64el-linux-gnuabi64.nix_2_4
2022-05-26 13:53:36 -07:00
Ben Siraphob
02dbca8c2b
Merge pull request #161270 from Stunkymonkey/darwin-phases 2022-04-30 10:58:05 -06:00
Artturin
33415135b0 config.contentAddressedByDefault: init option 2022-04-27 23:21:32 +03:00
Alyssa Ross
fd78240ac8
treewide: use lib.getLib for OpenSSL libraries
At some point, I'd like to make another attempt at
71f1f4884b ("openssl: stop static binaries referencing libs"), which
was reverted in 195c7da07d.  One problem with my previous attempt is
that I moved OpenSSL's libraries to a lib output, but many dependent
packages were hardcoding the out output as the location of the
libraries.  This patch fixes every such case I could find in the tree.
It won't have any effect immediately, but will mean these packages
will automatically use an OpenSSL lib output if it is reintroduced in
future.

This patch should cause very few rebuilds, because it shouldn't make
any change at all to most packages I'm touching.  The few rebuilds
that are introduced come from when I've changed a package builder not
to use variable names like openssl.out in scripts / substitution
patterns, which would be confusing since they don't hardcode the
output any more.

I started by making the following global replacements:

    ${pkgs.openssl.out}/lib -> ${lib.getLib pkgs.openssl}/lib
    ${openssl.out}/lib -> ${lib.getLib openssl}/lib

Then I removed the ".out" suffix when part of the argument to
lib.makeLibraryPath, since that function uses lib.getLib internally.

Then I fixed up cases where openssl was part of the -L flag to the
compiler/linker, since that unambigously is referring to libraries.

Then I manually investigated and fixed the following packages:

 - pycurl
 - citrix-workspace
 - ppp
 - wraith
 - unbound
 - gambit
 - acl2

I'm reasonably confindent in my fixes for all of them.

For acl2, since the openssl library paths are manually provided above
anyway, I don't think openssl is required separately as a build input
at all.  Removing it doesn't make a difference to the output size, the
file list, or the closure.

I've tested evaluation with the OfBorg meta checks, to protect against
introducing evaluation failures.
2022-03-30 15:10:00 +00:00
Felix Buehler
63a37b844c darwin: deprecate phases 2022-02-28 10:58:08 +01:00
zowoq
f5227f0643 bootstrap-tools: disable iconv for the hello test 2022-02-08 08:57:11 +10:00
Dmitry Kalinkin
7673650020
stdenv/darwin: fix for curl with zstd and idn2
The `curlMinimal` is to be used throughout the early bootstrap
stages. The final stage will allow the new references of the `curl`.

Fixes: 29526bc2 ('curl: IDN support requires libidn2 package')
2022-01-13 19:45:30 -05:00
toonn
5d23e617ec stdenv: Pass standalone argument for libcxxabi 2021-11-24 15:17:13 -08:00
toonn
63a3ead084 stdenv: Darwin no longer needs 10.11 workaround
Co-authored-by: happysalada <raphael@megzari.com>
2021-11-24 15:17:13 -08:00
toonn
77fc5f8c14 bootstrap-tools: Introduce getLib for lib outputs
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
2021-11-24 15:17:13 -08:00
toonn
cc767e1d05 darwin.stdenv: Update to LLVM 11 and clang 11.1.0 2021-11-24 15:17:13 -08:00
toonn
84454c3233 darwin.Libsystem: Patch TargetConditionals.h
`TargetConditionals.h` was missing several definitions, like
`TARGET_OS_TV` that are part of SDK 10.12 at least. And one that doesn't
seem to occur in any SDK afaict, `TARGET_OS_EMBEDDED_OTHER`.

I added the definitions from SDK 10.12 verbatim and defined
`TARGET_OS_EMBEDDED_OTHER` to be equal to `0`.

I think none of this works if `darwin.Libsystem` is used to build for
linux or iOS though so maybe this needs a more thorough fix?

This reverts 336d82617f because it's no
longer necessary.
2021-11-24 15:17:13 -08:00
toonn
e07eef85e8 xnu: Fix python3 patch
This reverts 7da313e10a because it's no
longer necessary.
2021-11-24 15:17:13 -08:00
Daiderd Jordan
7e4880c21b stdenv: update darwin bootstrap tools for llvm11
Build from bdffd0b276.

https://hydra.nixos.org/build/124753463
(cherry picked from commit f4a08349ed43a36ad5489a4ab548fcf0c129215d)
2021-11-24 15:17:13 -08:00
Emily
20a6d8a241 libxml2: use libiconv on Darwin
This fixes e.g. python3Packages.beautifulsoup4, which has tests relying
on the wider encoding support enabled by libiconv.

Fixes #137678.
2021-10-02 10:05:17 +09:00
Michael Stone
470466fdbd
sigtool: 4a3719b4 -> 2a13539d (#138453)
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-09-23 17:14:07 +02:00
Vladimír Čunát
f47ee76a6a
stdenv/darwin: switch to the official bootstrap URLs 2021-07-16 10:44:04 +02:00
Vladimír Čunát
d6a51653d7
stdenv/darwin: update bootstrap tools for x86_64-darwin
Updated python3 will block bootstrapping otherwise.

Verification instructions:
$ xdg-open https://hydra.nixos.org/build/144118577#tabs-details
$ nix build /nix/store/sh9cd0rarr8a2db1zdr4fzl4qsmvz5jp-stdenv-bootstrap-tools
$ cat /nix/store/sh9cd0rarr8a2db1zdr4fzl4qsmvz5jp-stdenv-bootstrap-tools/nix-support/hydra-build-products
$ nix hash-path /nix/store/9h4d7s313wv3gkfwi493yr1wvdsz9lf2-stdenv-bootstrap-tools/on-server/{sh,bzip2,mkdir,cpio}
sha256-igMAVEfumFv/LUNTGfNi2nSehgTNIP4Sg+f3L7u6SMA=
sha256-K3rhkJZipudT1Jgh+l41Y/fNsMkrPtiAsNRDha/lpZI=
sha256-VddFELwLDJGNADKB1fWwWPBtIAlEUgJv2hXRmC4NEeM=
sha256-SWkwvLaFyV44kLKL2nx720SvcL4ej/p2V/bX3uqAGO0=
$ nix hash-file /nix/store/9h4d7s313wv3gkfwi493yr1wvdsz9lf2-stdenv-bootstrap-tools/on-server/bootstrap-tools.cpio.bz2
sha256-b65dXbIm6o6s6U8tAiGpR6SMfvfn/VFcZgTHBetJZis=
2021-07-10 17:03:40 +02:00
Daiderd Jordan
40ffedfb38
darwin: expose bootstrapLlvmVersion as argument in the stdenv
This enables the bootstrap stdenv test to specify the actual llvm
of the newly generated build instread of assuming it's the same version
as the current stdenv.
2021-06-13 17:32:06 +02:00
Daiderd Jordan
bdffd0b276
Merge pull request #121055 from toonn/bootstrap-tools-bump
darwin bootstrap-tools bump LLVM to 11
2021-06-09 08:23:20 +02:00
toonn
c7b433acf9
bootstrap-tools: libclang-cpp already included
Changes to llvmPackages have caused the `libclang-cpp*.dylib` files to
be included in the `clang-unwrapped.lib` output. So we no longer need to
copy them from libclang.
2021-06-06 20:17:32 +02:00
toonn
336d82617f
bootstrap-tools: Patch TargetConditionals.h
`TargetConditionals.h` was missing several definitions, like
`TARGET_OS_TV` that are part of SDK 10.12 at least. And one that doesn't
seem to occur in any SDK afaict, `TARGET_OS_EMBEDDED_OTHER`.

I added the definitions from SDK 10.12 verbatim and defined
`TARGET_OS_EMBEDDED_OTHER` to be equal to `0`.

This is a modified version of a patch to avoid a stdenv rebuild.
2021-06-06 20:17:13 +02:00
toonn
fa49e70c1b
bootstrap-tools: Include libclang-cpp.11.1.dylib too
We need to include the library with the version suffix because
`libclang-cpp.dylib` is just a symlink.
2021-06-05 19:56:53 +02:00
toonn
d259fcc3e2
bootstrap-tools: Include libclang-cpp.dylib
Credits to @LnL7, this is based on commit
ae68f1c447d775e64d6e16a33f2affe591c419bf from PR #85151.
2021-06-05 19:56:53 +02:00
toonn
4a2698221b
bootstrap-tools: Update to LLVM 11 2021-06-05 19:56:53 +02:00
toonn
3b8782a806
bootstrap-tools: Add bootstrapFiles argument
I was having a hard time testing new bootstrapFiles because
`make-bootstrap-tools.nix` imports `pkgspath` but does not pass anything
but the current system.

This is merely for convenience and I'm not entirely certain it's a
sensible thing to do, maybe generating new bootstrapFiles while
overriding the current bootstrapFiles isn't something you're supposed to
do?
2021-06-05 19:56:53 +02:00
toonn
7da313e10a
bootstrap-tools: Fix xnu python3 patch
This patch only affects the xnu included in the bootstrap-tools to avoid
a stdenv rebuild.
2021-06-05 19:55:38 +02:00
happysalada
1c1c7685d9 stdenv.darwin: nixpkgs-fmt 2021-06-02 19:03:48 +09:00
Andrew Childs
38207735f4 darwin/make-bootstrap-tools: move "lib" from install name to rpath
The rpath structure for the bootstrap tools was reworked to minimize
the amount of rewriting required on unpack, but the test was not
updated to match the different structure.

Additionally [1] builds that use the bootstrap version of libc++
cannot find libc++abi if the reference includes the "lib"
component (ie, libc++ refers to libc++abi with
@rpath/lib/libc++abi.dylib).

[1] https://logs.nix.samueldr.com/nix-darwin/2021-05-18#4993282

Test failure observed on Hydra: https://hydra.nixos.org/build/143130126
2021-05-24 15:04:39 +09:00
Andrew Childs
5d22db3c5e
stdenv/darwin: add bootstrap tools for aarch64-darwin
Taken from PR #12355; other parts are still being discussed.
2021-05-18 21:34:57 +02:00
Andrew Childs
768aae66ef stdenv/darwin: Apple Silicon support 2021-05-17 00:27:02 +09:00
Andrew Childs
3eacdfe24a stdenv/darwin: bootstrap tools for Apple Silicon 2021-05-17 00:27:02 +09:00
John Ericson
7bba32a069 darwin packages: Get ready for cross
If things build fine with `stdenvNoCC`, let them use that. If tools
might be prefixed, prepare for that, either by directly splicing or just
using the env vars provided by the wrapper setup-hooks.

Co-authored-by: Dmitry Kalinkin <dmitry.kalinkin@gmail.com>
2021-05-11 16:07:01 -04:00
John Ericson
a3e54cb582 Merge remote-tracking branch 'upstream/staging-next' into staging 2021-05-06 15:48:25 -04:00
John Ericson
470640e7fe treewide: Do a number of no-op cleanups for cross and darwin
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.

In particular:

 - Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
   a proper `callPackage`.

 - Adjust Darwin `stdenv`'s overlays keeping things from the previous
   stage to not stick around too much.

 - Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
   closer to a unified LLVM and GCC bootstrap.
2021-05-06 11:17:26 -04:00
Andrew Childs
7869d16545 llvmPackages: Multuple outputs for everythting
Also begin to start work on cross compilation, though that will have to
be finished later.

The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.

Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.

----

Other misc notes, highly incomplete

- lvm-config-native and llvm-config are put in `dev` because they are
  tools just for build time.

- Clang no longer has an lld dep. That was introduced in
  db29857eb3, but if clang needs help
  finding lld when it is used we should just pass it flags / put in the
  resource dir. Providing it at build time increases critical path
  length for no good reason.

----

A note on `nativeCC`:

`stdenv` takes tools from the previous stage, so:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`

while:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
2021-04-30 05:41:00 +00:00
John Ericson
50a11f4f43
Merge pull request #120993 from regnat/easy-ca
Make the bootsrap respect the contentAddressedByDefault setting
2021-04-28 11:27:59 -04:00
regnat
14f66d60a7 Make the bootsrap respect the contentAddressedByDefault setting
Patch every `derivation` call in the bootsrap process to add it a
conditional `__contentAddressed` parameter.

That way, passing `contentAddressedByDefault` means that the entire
build closure of a system can be content addressed
2021-04-28 10:25:49 +02:00
Andrew Childs
44f09ccabf darwin: move deployment target and sdk version to platform config 2021-03-26 15:10:22 +09:00
Ryan Burns
bcd7b0dd5d darwin/make-bootstrap-tools: avoid brotli dependency
Needed as of e3d19670a0
2021-03-18 12:30:16 -07:00
Andrew Childs
21e3a8abe7 stdenv/darwin: allow brotli reference from curl
Brotli is recently a default dependency of curl in nixpkgs.

See e3d19670a0 in #112947
2021-03-09 18:24:01 +09:00
Ben Siraphob
aa8868c7cc pkgs/build-support: stdenv.lib -> lib 2021-01-24 19:07:54 -08:00
John Ericson
fa204a97d8 stdenv: Remove stdenv.platform
This was not working after #110544 as caught by @r-burns in
https://github.com/NixOS/nixpkgs/pull/110544#issuecomment-766444647.
Thankfully it isn't used anymore and I believe wasn't documented either.
(I at least did not remember it existed.)
2021-01-24 22:51:12 +00:00
Monson Shao
2467f5e4d2
darwin.apple-sdk: drop appleSdkVersion
In fact no one is using appleSdkVersion, and stdenv is decoupled with sdk.
2021-01-21 00:26:16 +08:00