Commit graph

3008 commits

Author SHA1 Message Date
Vincent Laporte
66efb76c75 ocamlPackages.buildDunePackage: fix meta.platforms 2018-11-30 16:19:03 +00:00
Charles Duffy
8f90b33240
buildMaven: Support metadata-only dependencies
The circumstances which can make this necessary are discussed in NixOS/mvn2nix-maven-plugin#17
2018-11-29 17:29:59 +00:00
Matthew Bauer
7fa8c41c4a
Merge pull request #50235 from illegalprime/more-fhsuserenv-blacklists
fix FHSUserEnv blacklists
2018-11-28 09:32:01 -06:00
Thomas Tuegel
fb7749620b
melpaBuild: Get package-build from melpa/package-build
This commit causes melpaBuild to use package-build from melpa/package-build
instead of melpa/melpa. Development of package-build happens in the former
repository whereas the latter is much larger, containing also the MELPA
recipes. We do not need to fetch the MELPA recipes from melpa/melpa, as we fetch
them one-by-one for Nixpkgs.
2018-11-27 09:36:28 -06:00
Thomas Tuegel
762295a39c
Merge pull request #51095 from bhipple/fix/emacs-builders
emacsPackagesNg.trivialBuild: cleanup and standardize function
2018-11-27 09:34:12 -06:00
Jörg Thalheim
afbdeb7b9b
Merge pull request #50802 from aszlig/autopatchelf-improvements
autoPatchelfHook: Fixes/improvements for Android SDK emulator
2018-11-27 10:25:26 +00:00
Jörg Thalheim
f12bd000b9
Merge pull request #49290 from krebs/nix-writers
get nix-writers into nixpkgs
2018-11-27 07:17:03 +00:00
lassulus
2d02cd7790 build-support writers: add tests 2018-11-27 07:08:13 +01:00
lassulus
abd0efae35 build-support: add writers from krebs/writers
Reference https://github.com/krebs/nix-writers revision 40fde9e
2018-11-27 07:08:12 +01:00
Benjamin Hipple
a7d1474023 emacsPackagesNg.trivialBuild: cleanup and standardize function
No real function change here, but this updates the trivial and melpa builders to
be formatted more consistently with the rest of the builders, and swaps
`eval "$preBuild"` for the more standard `runHook preBuild`.
2018-11-27 00:23:04 -05:00
aszlig
4a6e3e4185
autoPatchelfHook: Skip on missing segment headers
If the file in question is not a shared object file but an ELF, we
really want to skip the file, because we won't have anything to patch
there.

For example if the file is created via "gcc -c -o foo.o foo.c", we don't
get a segment header and so far autoPatchelf was trying to patch such a
file.

By checking for missing segment headers, we're now no longer going to
attempt patching such a file.

Signed-off-by: aszlig <aszlig@nix.build>
Reported-by: Sander van der Burg <svanderburg@gmail.com>
2018-11-26 01:58:36 +01:00
aszlig
9f23a63f79
autoPatchelfHook: Fix type of norecurse variable
While declaring it as an array doesn't do any harm in our usage, it
might be a bit confusing when reading the code.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-26 01:13:59 +01:00
aszlig
2faf905f98
autoPatchelfHook: Add addAutoPatchelfSearchPath
This function is useful if autoPatchelf is invoked during some of the
phases of a build and allows to add arbitrary shared objects to the
search path.

So far the same functionality was in autoPatchelf itself, but not
available as a separate function, so when adding shared objects to the
dependency cache one would have to do so manually.

The function also has the --no-recurse flag, which prevents recursing
into subdirectories.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-25 16:22:32 +01:00
Jörg Thalheim
c424c1161b
Merge pull request #50560 from oxij/pkgs/fetchurl-mirrors
fetchurl: mirrors: http -> https, https before http, http before ftp
2018-11-23 11:56:44 +00:00
Jan Malakhovski
8dcf831978 fetchurl: mirrors: http -> https, https before http, http before ftp
Because HTTP has a higher probability of working behind proxies.
2018-11-22 09:38:22 +00:00
Matthew Bauer
4e68511bb1 bintools: use i386 on all 32 bit x86 systems 2018-11-21 09:38:28 -06:00
Jörg Thalheim
952f4fda86
makeRustPlatform: refactor to make it easier to understand
It is now clearer what is supposed to be in the rust attribute set
without having studied type theory. The amount of code is identically.
2018-11-21 12:44:58 +00:00
aszlig
3ca35ce0b2
autoPatchelfHook: Add --no-recurse flag
This is to be used with the autoPatchelf command and allows to only
patch a specific file or directory without recursing into
subdirectories.

Apart from being able to run the command in a standalone way, as
detailled in the previous commit this is also needed for the Android SDK
emulator, because according to @svanderburg there are subdirectories we
don't want to patch.

The reason why I didn't use GNU getopt is that it might not be available
on all operating systems and the getopts bash builtin doesn't support
long arguments. Apart from that, the implementation for recognizing the
flag is pretty trivial and it's also using bash builtins only, so if we
want to do something really fancy someday, we can still change it.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-20 00:11:29 +01:00
aszlig
e4fbb244ee
autoPatchelfHook: Allow to prevent automatic run
If you want to only run autoPatchelf on a specific path and leave
everything else alone, we now have a $dontAutoPatchelf environment
variable, which causes the postFixup hook to not run at all.

The name "dontAutoPatchelf" probably is a bit weird in conjunction with
putting "autoPatchelfHook" in nativeBuildInputs, but unless someone
comes up with a better name I keep it that way because it's consistent
with all the other dontStrip, dontPatchShebangs, dontPatchELF and
whatnot.

A specific example where this is needed is when building the Android SDK
emulator, which contains a few ARM binaries in subdirectories that
should not be patched. If we were to run autoPatchelf on all outputs
unconditionally we'd run into errors because some ARM libraries couldn't
be found.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-20 00:07:38 +01:00
aszlig
d03e4ffdbf
autoPatchelfHook: Make easier to run autoPatchelf
The autoPatchelf main function which is run against all of the outputs
was pretty much tailored towards this specific setup-hook and was
relying on $prefix to be set globally.

So if you wanted to run autoPatchelf manually - let's say during
buildPhase - you would have needed to run it like this:

  prefix=/some/directory autoPatchelf

This is now more intuitive and all you need to do is run the following:

  autoPatchelf /some/directory

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-19 17:18:27 +01:00
Austin Seipp
04a543b3a0 defaultCrateOverrides: foundationdb native dependencies
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2018-11-17 19:28:48 -06:00
zimbatm
695a3d4254
Merge pull request #50302 from zimbatm/libredirect-misc
libredirect: misc changes
2018-11-14 00:16:56 +01:00
zimbatm
d04a1265a1
libredirect: set install_name on Darwin
fixes https://github.com/NixOS/nixpkgs/pull/50246#issuecomment-437975038
2018-11-14 00:05:26 +01:00
zimbatm
91c130e2f5
libredirect: introduce optional setup-hook
This allows to simplify the usage of libredirect inside of nix build
sandboxes. Add "libredirect.hook" to the build inputs to get everything
linked in automaticall. All that's left is to set NIX_REDIRECTS and call
the target program.
2018-11-14 00:05:23 +01:00
Matthew Bauer
4b8c1d23d0
Merge pull request #42794 from telent/make-closure-x
make-closure: needs build system mkdir and jq
2018-11-13 15:02:08 -06:00
zimbatm
e62db105c4
libredirect: specify libName
reduces a bit of duplication and can also be used from the outside:

   export LD_PRELOAD=${libredirect}/lib/${libredirect.libName}
2018-11-13 12:26:15 +01:00
Antoine Eiche
c12f75649e dockerTools.buildImageWithNixDb: simplifications and switch to closureInfo
Since Nix 2 is now the stable Nix version, we can use closureInfo
which simplifies the Nix database initialisation (size and hash are
included in the "dump").
2018-11-12 18:30:53 +01:00
aszlig
a815f53c60
libredirect: Add preload wrapper for stat()
Pull request #50246 was merged a bit too quickly and it was supposed to
fix libredirect on Darwin. However it still failed on Darwin and this
was missed by the person merging the pull request.

The reason this was failing was that there is no __xstat* on Darwin.

So I'm adding a wrapper for stat() as well as it works on Darwin and it
still doesn't hurt on GNU/Linux.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra, @zimbatm
2018-11-12 13:31:43 +01:00
aszlig
34dd1c68f8
libredirect: Add a small test
This is just a sanity check on whether the library correctly wraps the
syscalls and it's using the "true" executable for posix_spawn() and
execv().

The installCheckPhase is not executed if we are cross-compiling, so this
shouldn't break cross-compilation.

One thing I'm not actually sure is whether ${coreutils}/bin/true is
universally available on all the platforms, nor whether all the
functions we use in the test are available, but we can still fix that
after we've found out about that.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-12 11:02:54 +01:00
aszlig
ba1fddb315
libredirect: Use extensions.sharedLibrary
This is to make sure we get the correct shared library suffix of the
target platform. While for example on Darwin it would even work with the
hardcoded .so prefix it's IMHO a bit nicer to have the actual native
extension.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-12 10:08:02 +01:00
zimbatm
9ef52352bd
assume that it works on all unix platforms 2018-11-12 00:09:36 +01:00
zimbatm
d76ec523bb
use for cross-compilation 2018-11-12 00:08:18 +01:00
aszlig
753743c37b
libredirect: Add support for Darwin
The library can be used also on Darwin using it like this:

  NIX_REDIRECTS='foo=bar' \
  DYLD_INSERT_LIBRARIES=${libredirect}/lib/libredirect.so \
  DYLD_FORCE_FLAT_NAMESPACE=1 \
  some_program

So let's actually not hardcade gcc and add Darwin to meta.platforms.

No other changes seem to be required.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-11 19:29:12 +01:00
Michael Eden
a3488fb9ac fix FHSUserEnv blacklists 2018-11-11 10:32:09 -05:00
Théo Zimmermann
742bce7793
buildDunePackage: inline dune.installPhase for easier overriding
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2018-11-07 10:08:08 +01:00
Théo Zimmermann
406405d8bd
buildDunePackage: add support for pre and post phase hooks
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2018-11-07 10:08:08 +01:00
Théo Zimmermann
794158fcd5
buildDunePackage: new support function; use it to refactor some OCaml derivations 2018-11-07 10:08:03 +01:00
Yegor Timoshenko
77dad17ab6
Merge pull request #49725 from pbogdan/chrootenv-strip
chrootenv: strip the binary
2018-11-05 18:57:28 +00:00
Yegor Timoshenko
cea0e9226f
chrootenv: use meson 2018-11-04 11:33:34 +00:00
Piotr Bogdan
ccb76eeb3c chrootenv: strip the binary 2018-11-04 03:43:22 +00:00
aszlig
c64624b843
autoPatchelfHook: Correctly detect PIE binaries
I originally thought it would just be enough to just check for an INTERP
section in isExecutable, however this would mean that we don't detect
statically linked ELF files, which would break our recent improvement to
gracefully handle those.

In theory, we are only interested in ELF files that have an INTERP
section, so checking for INTERP would be enough. Unfortunately the
isExecutable function is already used outside of autoPatchelfHook, so we
can't easily get rid of it now, so let's actually strive for more
correctness and make isExecutable actually match ELF files that are
executable.

So what we're doing instead now is to check whether either the ELF type
is EXEC *or* we have an INTERP section and if one of them is true we
should have an ELF executable, even if it's statically linked.

Along the way I also set LANG=C for the invocations of readelf, just to
be sure we don't get locale-dependent output.

Tested this with the following command (which contains almost[1] all the
packages using autoPatchelfHook), checking whether we run into any
library-related errors:

  nix-build -E 'with import ./. { config.allowUnfree = true; };
    runCommand "test-executables" {
      drvs = [
        anydesk cups-kyodialog3 elasticsearch franz gurobi
        masterpdfeditor oracle-instantclient powershell reaper
        sourcetrail teamviewer unixODBCDrivers.msodbcsql17 virtlyst
        vk-messenger wavebox zoom-us
      ];
    } ("for i in $drvs; do for b in $i/bin/*; do " +
       "[ -x \"$b\" ] && timeout 10 \"$b\" || :; done; done")
  '

Apart from testing against library-related errors I also compared the
resulting store paths against the ones prior to this commit. Only
anydesk and virtlyst had the same as they didn't have self-references,
everything else differed only because of self-references, except
elasticsearch, which had the following PIE binaries:

  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/autoconfig
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/autodetect
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/categorize
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/normalize

These binaries were now patched, which is what this commit is all about.

[1]: I didn't include the "maxx" package (MaXX Interactive Desktop)
     because the upstream URLs are no longer existing and I couldn't
     find them elsewhere on the web.

Signed-off-by: aszlig <aszlig@nix.build>
Fixes: https://github.com/NixOS/nixpkgs/issues/48330
Cc: @gnidorah (for MaXX Interactive Desktop)
2018-11-03 08:07:42 +01:00
Kristoffer Søholm
5e5e57c572 buildFHSUserEnv: use runScript in env (#49077)
This makes its behaviour conform to what is implied in the
documentation.
2018-10-30 22:47:08 +01:00
Michał Janiszewski
3f05186984 Compare to None using identity is operator
This is a trivial change that replaces `==` operator with `is` operator, following PEP 8 guideline:

> Comparisons to singletons like None should always be done with is or is not, never the equality operators.

https://legacy.python.org/dev/peps/pep-0008/#programming-recommendations
2018-10-30 21:30:56 +01:00
Matthew Bauer
412093994b gcc: support avr
- respect libc’s incdir and libdir
- make non-unix systems single threaded
- set LIMITS_H_TEST to false for avr
- misc updates to support new libc’s
- use multilib with avr

For threads we want to use:
- posix on unix systems
- win32 on windows
- single on everything else

For avr:
- add library directories for avrlibc
- to disable relro and bind
- avr5 should have precedence over avr3 - otherwise gcc uses the wrong one
2018-10-29 14:34:09 -05:00
Matthew Bauer
d59a9ac7cf avr: use new compilation infrastructure
Gets rid of:
  avrbinutils
  avrgcc

to replace with:
  pkgsCross.avr.buildPackages.binutils
  pkgsCross.avr.buildPackages.gcc
2018-10-29 14:34:09 -05:00
Jörg Thalheim
96c627b3f6
defaultCrateOverrides: add serde_derive 2018-10-28 21:59:19 +00:00
Jörg Thalheim
e0a5689528
defaultCrateOverrides: order alphabetically 2018-10-28 21:55:26 +00:00
Pierre-Etienne Meunier
ae3b4655a4 Carnix: 0.7.2 -> 0.8.10 (#40587)
Carnix: splits input into two parts: creates from creates.io and local ones
2018-10-28 00:06:29 +01:00
Jörg Thalheim
f10b935f84
breakpointHook: add for debugging failing builds
Usuage: Add breakpointHook to your `buildInputs` like this:

  stdenv.mkDerivation rec {
    # ...
    buildInputs = [ breakpointHook ];
  });

When the build fails as show in this example:

  pkgs.hello.overrideAttrs (old: {
    buildInputs = [ breakpointHook ];
    postPatch = ''
      false
    '';
  });

It will halt execution printing the following message:

build failed in patchPhase with exit code 1
To attach to this build run the following command as root:

   cntr attach -t command cntr-/nix/store/ynyb4n82x2r7sldd58pbb405jdqh5f00-hello-2.10

Installing cntr and running the command will provide shell access to the
build sandbox of failed build:

sudo cntr attach -t command cntr-/nix/store/ynyb4n82x2r7sldd58pbb405jdqh5f00-hello-2.10
WARNING: bad ownership on /nix/var/nix/profiles/per-user/root, should be 1000
[nixbld@localhost:/var/lib/cntr]$

At /var/lib/cntr the sandbox filesystem is mounted. All commands and
files of the system are still accessible within the shell.
To execute commands from the sandbox use the `cntr exec` subcommand.
2018-10-25 10:19:41 +01:00
Frederik Rietdijk
821a3beb10
Merge pull request #48306 from NixOS/staging-next
Merge staging-next into master
2018-10-18 11:23:04 +02:00