Commit graph

250 commits

Author SHA1 Message Date
Matthew Bauer
37273afd27
Merge pull request #43224 from volth/patch-179
[staging] substitute(): --subst-var was silently coercing to "" if the variable does not exist.
2018-07-21 15:48:14 -04:00
aszlig
739c835515
stdenv-setup: Remove superfluous check for /bin/sh
The line was essentially checking whether /bin/sh exists and is
executable and if that's the case, the isScript function returns
successfully.

When asking the author of this line on IRC it seems that even they can't
remember or imagine what this was supposed to be.

In summary: Whenever /bin/sh doesn't exist during a build, *any* file
given to isScript is reported as being a script even if it isn't.

This is kinda counter-intuitive and not something what somebody would
expect from a function called "isScript".

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra
2018-07-09 20:43:34 +02:00
volth
0de0ce5893 substitute(): --subst-var was silently coercing to "" if the variable does not exist. 2018-07-08 16:32:03 +00:00
John Ericson
5e17335bd7 Merge remote-tracking branch 'upstream/staging' into strictDeps 2018-05-14 23:33:03 -04:00
John Ericson
330ca731e8 treewide: Get rid of all uses of crossConfig
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)

Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:

 - bintools-wrapper and cc-wrapper
2018-05-14 23:30:37 -04:00
John Ericson
4f7cdd35d5
Merge pull request #40139 from obsidiansystems/modular-setup-hooks
treewide: Modular setup hooks
2018-05-07 15:32:10 -04:00
John Ericson
34a3233a2e stdenv: Support concatenating setup hooks from multiple parts. 2018-05-07 14:43:22 -04:00
Jan Malakhovski
50af975d85 stdenv: implement checkTarget and installCheckTarget autodetection 2018-04-25 19:53:25 +00:00
Jan Malakhovski
e9e06888ed stdenv: generic/setup.sh: cleanup installPhase 2018-04-25 19:53:25 +00:00
Jan Malakhovski
ad98c36f1b stdenv: generic/setup.sh: simplify buildPhase Makefile check 2018-04-25 19:53:24 +00:00
Shea Levy
b24ce2ae63
Handle sourceRoots with leading dashes 2018-02-15 11:50:24 -05:00
Shea Levy
f83b6e1130
unpackPhase: Handle sources starting with a hyphen 2018-01-24 21:58:57 -08:00
adisbladis
c2316114bc
stdenv: Kill off ensureDir 2018-01-09 11:14:48 +08:00
John Ericson
469fd89832 stdenv-setup: Ease the transition with native builds
- All deps go on the PATH

 - CC and Bintools wrappers with their host != depender's host still get their
   setup hooks run.

 - Environment hooks get applied to all packages

This isn't so elegent, but eases the transition on a very significant
PR.
2017-12-30 22:04:23 -05:00
John Ericson
a036473a0a {bintools,cc}-wrapper: Fix setup hook to respect the role of the cc-compiler
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
2017-12-30 22:04:21 -05:00
John Ericson
7f3ca3e21a stdenv: Fix handling of dependencies and hooks
4 far-reaching changes: Smaller PATH, New vars, different propagation
logic, and different hook logic

Smaller PATH
------------

`buildInputs` no longer go on the PATH at build time, as they cannot be
run when cross compiling and we don't want to special case. Simply make
a `nativeBuildInput` too if one needs them on the PATH. Fixes #21191.

Many new depedendency variables
-------------------------------

See the stdenv chapter of the nixpkgs manual. I pulled out the existing
documentation of dependency specification into a new section, and added
language for these two (and their propagated equivalents) along side
the others'.

More complex propagation logic
------------------------------

Before a propagated*XXX*Input always acted as if it was specified
directly as a *XXX*Input downstream. That's simple enough, but violates
the intended roles of each sort of dep, which has functional and not
just stylistic consequences.

The new algorithm is detailed in the manual, and ensures everything
ends up in the right place. I tried to give both an informal and formal
description, but I suspect in practice it will not make much sense
until one tries cross compiling, after which it will immediately make
sense as the only sane option.

Simplified hook logic
---------------------

Rather than `envHook` and `crossEnvHook`, whose behavior differs
depending on whether we are cross compiling or not, there is now one
hook per sort (or rather non-propagated and propagated pair of sorts)
of dependency. These new hooks have the same meaning regardless of
cross compilation. See the setup hook section of stdenv chapter of the
Nixpkgs manual for more details.
2017-12-30 22:04:21 -05:00
John Ericson
da19c34d0f stdenv setup: Always use both propagated files
This continues #23374, which always kept around both attributes, by
always including both propagated files: `propgated-native-build-inputs`
and `propagated-build-inputs`. `nativePkgs` and `crossPkgs` are still
defined as before, however, so this change should only barely
observable.

This is an incremental step to fully keeping the dependencies separate
in all cases.
2017-11-21 10:44:44 -05:00
John Ericson
6a5cda5131 stdenv setup: Run setup hooks and other processing after accumulating deps
I find the separation of concerns, accumulating, then processing, easier
to follow. Also, with my yet-to-be-merged cross work, the accumulation
part will become more complex.
2017-11-15 18:51:06 -05:00
John Ericson
f6fcb9bc0d stdenv: Turn on set -x if NIX_DEBUG >= 6
Why 6? It seems a decently high number, giving us room for more degrees
of debugging before the `set -x` sledgehammer without incurring a
mass-rebuild.
2017-09-26 11:24:19 -04:00
John Ericson
127a5f3357 treewide: Use (( "${NIX_DEBUG:-0}" >= 1) )) consistently 2017-09-26 11:24:19 -04:00
Eelco Dolstra
0061fae2e6 genericBuild: Communicate the current build phase to Nix
This allows the progress bar to show e.g.

  [1/9/59 built] building bison-3.0.4 (configurePhase): checking for strdup... yes
2017-09-07 22:24:27 +02:00
Eelco Dolstra
6b3cef2246 Remove tracePhases
This has not been used in a long time.
2017-09-07 22:15:37 +02:00
Orivej Desh
f4044c1ccc stdenv-setup: list environment variables with awk 2017-09-03 12:57:08 +00:00
Orivej Desh
a09d9e7cd4 stdenv-setup: fix substituteAll with set -eu
Environment variable filter in substituteAll was not precise and produced
undefined and invalid variable names.  Vladimír Čunát tried to fix that in [1],
but `env -0` did not work during Darwin bootstrap, so [2] reverted this change
and replaced an error due to invalid variables with a warning.  Recently in #28057
John Ericson added `set -u` to `setup.sh` and undefined variables made the setup
fail during e.g. `nix-build -A gnat` with `setup: line 519: !varName: unbound
variable`.

[1] 62fc8859c1
[2] 81df035429
2017-09-03 12:57:08 +00:00
John Ericson
81194eef45 stdenv-setup: Use set -u as much as possible
Older bash version, like those in the bootstrap tools and on macOS,
currently confuse variables defined as an empty array with undefined
variables. `${foo+"${foo[@]}"}` will prevent `set -u` problems with
empty arrays and older without making a single '' in the empty case.

Care is taken to `set +u` when running hooks so as to not break existing
packages.
2017-08-23 15:57:56 -04:00
Tuomas Tynkkynen
7320fa9d45 Revert "stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs"
This reverts commit eeabf85780.

This change suddenly makes tons of stdenv internals visible in
nativeBuildInputs of every derivation, which doesn't seem desirable.
E.g:

````
nix-repl> hello.nativeBuildInputs
[ «derivation /nix/store/bcfkyf6bhssxd2vzwgzmsbn7b5b9rpxc-patchelf-0.9.drv»
  «derivation /nix/store/4wnshnz9wwanpfzcrdd76rri7pyqn9sk-paxctl-0.9.drv»
  << snip 10+ lines >>
  «derivation /nix/store/d35pgh1lcg5nm0x28d899pxj30b8c9b2-gcc-wrapper-6.4.0.drv»
]
````
2017-08-18 13:21:56 +03:00
John Ericson
eeabf85780 stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs
Additionally, instead of pulling them from `setup.sh`, route them via
Nix. This gets us one step closer to making stdenv be a plain attribute
set instead of a derivation.
2017-08-15 18:24:54 -04:00
Linus Heckemann
17753fa005 stdenv: fix typo in setup.sh 2017-08-09 17:33:02 +01:00
John Ericson
42f35503b5 cc-wrapper: Make hygienic
See the added comments for what exactly has been done.
2017-08-07 03:05:50 -04:00
John Ericson
820e4021d3 stdenv-setup: Remove any declare -g
This is invalid before bash-4.2, affecting bash used impurely in
nix-shell on MacOS.
2017-07-26 09:11:18 -04:00
John Ericson
ea7d13cf1a stdenv-setup and misc hooks: Work with bash-3.4 for MacOS nix-shell
This is a temporary measure until this impurity is removed from Nix.
2017-07-26 09:08:01 -04:00
John Ericson
f6f40e3fe5 stdenv-setup and misc pkgs: Revert to space-deliminated propagated-* files
We cannot switch to line-delimited yet, because certain Nix commands do
not read in the entire file, but just the first line.
2017-07-26 09:07:55 -04:00
John Ericson
34c0ba498c stdenv-setup: Add quotes that don't do anything for consistency.
@vcunat and others rightly point out that it's easier to quote always,
than learn Bash's idiosyncrasies enough to know when it doesn't make a
difference.

This reverts commit 2743078f66, which
removes quotes that don't do anything, and then goes further adding
even more quotes.
2017-07-25 14:36:00 -04:00
Eelco Dolstra
6669a3b477
stdenv: Remove log nesting
Nix/Hydra no longer support pretty printing of logs, so this is no
longer useful.
2017-07-24 14:30:46 +02:00
John Ericson
30a1420414 stdenv-setup: Pull out and explain 3-part printing of commands
@Dezgeg made the good point that the reasons for doing this were not at
all intuitive.
2017-07-13 16:31:39 -04:00
John Ericson
2743078f66 stdenv-setup: Remove useless quotes
foo=$1 surprisingly doesn't need quotes in Bash. Word splits are only
syntactic in string variable (not array var!) assignments.
2017-07-13 14:59:53 -04:00
John Ericson
273a4c1c78 stdenv-setup: Combine [[ .. ]] && [[ .. ]] into one [[ .. && .. ]]
Also remove useless quotes on same line
2017-07-13 14:57:16 -04:00
John Ericson
5d693c84d2 stdenv-setup: Clean up 'substitute()' for style and error handling
It now blows up on null byte in file (rather than silently truncating),
and invalid arguments (rather than silently skipping).
2017-07-12 17:47:20 -04:00
John Ericson
5d4efb2c81 stdenv-setup: Misc improvements as directed by ShellCheck
I took some liberties with the flags-echoing code to make it more
concise and correct. Also, a few warnings in findInputs and friends I
skipped because I am going to rewrite those anyways.

Thanks @grahamc for telling me about this great linter!
2017-07-12 15:31:10 -04:00
John Ericson
8d76effc17 stdenv-setup: Make the package accumulators associative arrays instead of strings
This is generally cleaner: less eval, less worrying about separators,
and probably also faster. I got the idea from that python wrapper
script.
2017-07-12 15:30:56 -04:00
John Ericson
3cb745d5a6 stdenv: Store one package per line in nix-support/propagated-*
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.

Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.

Two misc notes:

 - `propagated-user-env-packages` also needed to be adjusted as
   sometimes it is copied to/from the propagated input files.

 - `local fd` should ensure that file descriptors aren't clobbered
   during recursion.
2017-07-10 13:32:13 -04:00
John Ericson
e57a220f81 stdenv, swift: Use local fd in is* bash functions for hygiene 2017-07-07 11:40:07 -04:00
John Ericson
a14cf06182 stdenv: Harden hook runners
Instead of eval, use a "nameref" to get the name of the array and
iterate with that. Also, make the for-loop parameter a local variable,
too.
2017-07-07 11:35:09 -04:00
Eelco Dolstra
94d164dd7f
Add a setup hook for detecting $TMPDIR references in RPATHs and wrapper scripts 2017-05-04 20:23:57 +02:00
Tuomas Tynkkynen
ce56c99edc mkDerivation: Don't pass buildInputs to stdenv builder in nativeBuildInputs
When not cross compiling, nativeBuildInputs and buildInputs have
identical behaviour. Currently that is implemented by having
mkDerivation do a concatenation of those variables in Nix code and pass
that to the builder via the nativeBuildInputs attribute.

However, that has some annoying side effects, like `foo.buildInputs`
evaluating to `[ ]` even if buildInputs were specified in the nix
expression for foo.

Instead, pass buildInputs and nativeBuildInputs in separate variables as
usual, and move the logic of cross compilation vs. native compilation to
the stdenv builder script. This is probably a tiny bit uglier but
fixes the previous problem.

Issue #4855.
2017-03-02 03:26:48 +02:00
Vladimír Čunát
af38c05587 stdenv stripHash(): fixup after #19324 2016-10-12 23:45:30 +02:00
Profpatsch
bef6bef0d2
stdenv/stripHash: print to stdout, not to variable
`stripHash` documentation states that it prints out the stripped name to
the stdout, but the function stored the value in `strippedName`
instead.

Basically all usages did something like
`$(stripHash $foo | echo $strippedName)` which is just braindamaged.
Fixed the implementation and all invocations.
2016-10-11 18:34:36 +02:00
Nikolay Amiantov
adaee7352b stdenv: leave SSL_CERT_FILE in shells (#15571) 2016-09-01 20:50:08 +02:00
Eelco Dolstra
8a84fc0217 Tweak error message 2016-08-26 18:58:49 +02:00
Robin Gloster
b6c204f088
stdenv substitute: fail on non-existant input file
fixes #9744
2016-08-26 16:27:36 +00:00