Commit graph

56 commits

Author SHA1 Message Date
John Ericson
38ebb8ff82 fetchurl: Eliminate pointless cross differences 2019-11-11 00:25:24 -05:00
Eelco Dolstra
267c8d6b2f
fetchurl (and derived functions): Support SRI hashes
E.g.

  fetchFromGitHub {
    owner = "NixOS";
    repo = "nix";
    rev = "ad42a784690449873fccb20192bd2150da81c56d";
    hash = "sha256-ZXeadXUJMXV5lSLz6TOBeL/SSOVwQ8ywxU5AFMCnbRU=";
  }
2019-07-11 15:52:35 +02:00
Jörg Thalheim
eac6797380
prefer-fetch-remote: an overlay to fetch on remote builders
This is useful when running tools like NixOps or nix-review
on workstations where the upload to the builder is significantly
slower then downloading the source on the builder itself.
2019-01-18 14:41:10 +00:00
Yegor Timoshenko
f44012ba10
lib.release -> lib.trivial.release 2018-06-11 02:14:18 +00:00
Yegor Timoshenko
5a232b0f5b
Leverage lib.release 2018-06-11 01:50:06 +00:00
Yegor Timoshenko
16ed09a10e
Merge branch 'master' into fetchurl-user-agent 2018-05-18 10:39:44 +03:00
Jan Malakhovski
a89899ce4e fetchurl: cleanup, better errors
Also fix what seems like bugs in uncommon `stdenv`s.
2018-02-18 14:24:53 +00:00
John Ericson
940c4fa3f5 treewide: Fetchers should use stdenvNoCC. 2018-01-10 11:18:44 -05:00
John Ericson
3d59b4d285 treewide: Fixed output fetch* derivations should use nativeBuildInputs 2018-01-09 20:14:46 -05:00
Dan Peebles
0cb623c3d9
fetchurl: add user agent
It would be nice to be able to track Nix requests. It's not trustworthy,
but can be helpful for stats and routing in HTTP logs.

Since `fetchurl` is used so widely, we should "magically" get a UA on
`fetchzip`, `fetchFromGitHub`, and other related fetchers.

Since `fetchurl` is only used for fixed-output derivations, this should
cause no mass rebuild.

User-Agent example: curl/7.57.0 Nixpkgs/18.03
2017-12-23 22:20:56 +00:00
Frederik Rietdijk
ef21b240c8 fetchurl: add passthru
so that we could add e.g. the `version` attribute.
2017-11-05 10:12:19 +01:00
Shea Levy
b71b7ee622 fetchFromGitHub: Allow private repos, hosted githubs 2017-05-19 15:38:01 -04:00
Sven Slootweg
fa8adf2c47 fetchurl: fixed typo in error message
This typo was likely introduced by copy-pasting the error message from elsewhere and forgetting to change the text, during the MD5 deprecation process (#4491).
2017-04-21 16:41:09 +02:00
Robin Gloster
f57185db95
fetch-*: remove md5 support
fixes #4491
2017-03-20 22:23:41 +01:00
Eelco Dolstra
e3a873479e
Remove fetchMD5warn
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.

This also reverts commit 2ca8833383.
2016-11-30 15:14:30 +01:00
Michael Raskin
f603dc11a6 fetch*: print a trace warning about md5 deprecation 2016-10-09 16:19:04 +02:00
Profpatsch
61462c94e6 lib/fetchers.nix: factor out impure proxy vars (#18702)
Apparently everyone just copied those variables, instead of creating a
library constant for them. Some even removed the comment. -.-
2016-09-17 21:50:01 +02:00
Domen Kožar
0f9268e52c fetchurl: assert required Nix version for sha512 2016-04-14 12:50:21 +01:00
Eelco Dolstra
3ecbe604ef fetchurl: Support SHA-512 hashes 2016-04-13 14:11:14 +02:00
Scott Olson
43a523526d Require at least one of url or urls in fetchurl. 2016-02-19 03:18:21 -06:00
Scott Olson
9cf93ba135 Simplify fetchurl assertion logic.
The two lines I removed technically assert the exact same thing, since `!a -> b`
is equivalent to `a || b`. So, I replaced the two lines with the more symmetric
form to make it clearer.
2016-02-18 22:39:43 -06:00
Domen Kožar
c7383cb34b fetchurl: support executables 2016-01-19 11:17:49 +01:00
Jan Malakhovski
bdf32ed2ab fetchurl: allow adding meta info; fetchFrom*: add meta.homepage
The point of this is to be able to do `meta.homepage = src.meta.homepage;`
instead of the usual copy-paste for the packages that are hosted
on these hosting services.
2015-05-24 14:26:39 +00:00
Vladimír Čunát
c00d36f6c1 fetchurl: fix typo from 105154afed pushed today 2015-02-22 20:27:29 +01:00
Vladimír Čunát
105154afed fetchurl: build the mirror list locally 2015-02-22 19:26:22 +01:00
Domen Kožar
909b62fa6e fetchurl: friendly message if hash is not given 2014-11-23 17:49:32 +01:00
Linquize
f40db99c7b fetchurl: Fix typo in comment (close #2591) 2014-05-13 11:23:50 +02:00
Eelco Dolstra
ea36f3b868 fetchFromGitHub: Use .tar.gz instead of .zip
Also clean up the name attribute of fetchzip derivations a bit.
2014-05-09 15:53:44 +02:00
Eelco Dolstra
c8df888858 Add a function "fetchzip"
This function downloads and unpacks a file in one fixed-output
derivation. This is primarily useful for dynamically generated zip
files, such as GitHub's /archive URLs, where the unpacked content of
the zip file doesn't change, but the zip file itself may (e.g. due to
minor changes in the compression algorithm, or changes in timestamps).

Fetchzip is implemented by extending fetchurl with a "postFetch" hook
that is executed after the file has been downloaded. This hook can
thus perform arbitrary checks or transformations on the downloaded
file.
2014-05-08 15:30:17 +02:00
Eelco Dolstra
83ca2c272a fetchurl: Ensure that ‘urls’ is a list 2014-02-19 13:58:42 +01:00
Eelco Dolstra
acba9240cd nixos.org/tarballs -> tarballs.nixos.org
It's currently the same machine, but tarballs.nixos.org should become
an S3/CloudFront site eventually.
2013-06-25 14:12:16 +02:00
Gergely Risko
07b26ce493 fetchUrl: add curlOpts parameter (merge #535) 2013-05-16 10:45:16 +02:00
Eelco Dolstra
04765da09a fetchurl: Don't wait too long for the hashed mirrors
If the hashed mirror (nixos.org/tarballs) doesn't respond in 15
seconds, give up and try the original URL.
2013-02-06 15:15:28 +01:00
Eelco Dolstra
f337ab4430 fetchurl: Remove some compatibility hacks 2013-02-06 15:00:33 +01:00
Eelco Dolstra
1baf48f087 Do some let-floating to prevent repeated evaluation of impureEnvVars 2012-08-13 15:15:16 -04:00
Eelco Dolstra
aded38809b * Turn on `preferLocalBuild' for fetchurl.
svn path=/nixpkgs/trunk/; revision=22948
2010-08-04 12:37:03 +00:00
Yury G. Kudryashov
952bc80b4d Let fetchurl retry failed downloads. Add NIX_CURL_FLAGS impure var
svn path=/nixpkgs/trunk/; revision=22006
2010-05-27 18:59:19 +00:00
Yury G. Kudryashov
796ffd9f71 Move "fetchurl restricted" code to requireFile.
This reverts r21474 and r21467.
Also migrate wtk (r21472) to new API.

svn path=/nixpkgs/trunk/; revision=21575
2010-05-03 09:13:17 +00:00
Yury G. Kudryashov
3e1e56e10c fetchurl restricted: list nix-prefetch-url as well
svn path=/nixpkgs/trunk/; revision=21474
2010-05-01 07:07:36 +00:00
Yury G. Kudryashov
404c314412 Add support for restricted downloads
svn path=/nixpkgs/trunk/; revision=21467
2010-04-30 20:40:42 +00:00
Eelco Dolstra
18a08aa008 * Add an openSUSE mirror that contains discontinued releases (such as
10.3).

svn path=/nixpkgs/trunk/; revision=18385
2009-11-17 14:26:42 +00:00
Marc Weber
ec0e356b9a Revert "adding NIX_CONTINUE_DOWNLOADS feature, see mkdir comment"
This reverts commit -r16401

svn path=/nixpkgs/trunk/; revision=16405
2009-07-16 17:17:23 +00:00
Marc Weber
10f6c2592c adding NIX_CONTINUE_DOWNLOADS feature, see mkdir comment
svn path=/nixpkgs/trunk/; revision=16401
2009-07-16 15:18:26 +00:00
Eelco Dolstra
aa84bce64d * Get rid of all references to nix.cs.uu.nl.
svn path=/nixpkgs/trunk/; revision=13299
2008-11-14 16:57:19 +00:00
Eelco Dolstra
eebb241519 * Write the list of mirrors to a file that we can reuse between
fetchurl instantiations, instead of passing the mirrors to fetchurl
  instantiations via environment variables.  This makes the resulting
  store derivations (.drv files) much smaller, which in turn makes
  nix-env/nix-instantiate faster (4.8 -> 4.2 seconds on nix-env -qa
  --out-path).

svn path=/nixpkgs/trunk/; revision=12695
2008-08-22 15:53:21 +00:00
Eelco Dolstra
a7c209e474 * Interface to support mirror:// in nix-prefetch-url. The function
resolveMirrorURLs produces a file containing the URLs to which a
  mirror:// URL resolves.

svn path=/nixpkgs/trunk/; revision=12410
2008-07-23 16:04:10 +00:00
Eelco Dolstra
76a8d120a4 * Allow the user to override the list of mirrors for specific
mirror:// sites through environment variables, e.g.

  NIX_MIRRORS_gnu="ftp://ftp.nluug.nl/pub/gnu/ ftp://ftp.gnu.org/pub/gnu/"

  or

  NIX_MIRRORS_sourceforge="http://surfnet.dl.sourceforge.net/sourceforge/"

svn path=/nixpkgs/trunk/; revision=9302
2007-09-11 15:00:49 +00:00
Eelco Dolstra
6c4fd2e3df * Allow the user to override the list of content-addressable mirrors
from the command-line by setting the NIX_HASHED_MIRRORS environment
  variable.

svn path=/nixpkgs/trunk/; revision=9301
2007-09-11 13:48:53 +00:00
Eelco Dolstra
45a2c87402 * Support for mirror:// URLs a la Gentoo (NIXPKGS-70). Instead of
fetchurl {
      url = http://heanet.dl.sourceforge.net/sourceforge/zapping/zapping-0.9.6.tar.bz2;
      md5 = "8306775c6a11de4d72345b5eee970ea6";
    };

  you can write

    fetchurl {
      url = mirror://sourceforge/zapping/zapping-0.9.6.tar.bz2;
      md5 = "8306775c6a11de4d72345b5eee970ea6";
    };

  which causes fetchurl to try the SourceForge mirrors listed in the
  `sourceforge' attribute in build-support/fetchurl/mirrors.nix.
  (They're currently tried in sequence, and the lists of mirrors are
  not configurable yet.)

  The syntax for mirror URLs is mirror://site/path/to/file, where
  `site' is currently one of `sourceforge', `gnu' (mirrors of
  ftp://ftp.gnu.org/pub/gnu) and `kernel' (mirrors of
  http://www.all.kernel.org/pub/).

svn path=/nixpkgs/trunk/; revision=9197
2007-08-27 12:44:01 +00:00
Eelco Dolstra
50321b735b * Basic transparent mirror support in fetchurl (NIXPKGS-70). When
fetching a file with hash HASH of type TYPE, we first try to
  download <base-url>/<type>/<hash>, where <base-url> is one of a list
  of mirrors.  For instance, given

    src = fetchurl {
      url = http://releases.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.6/source/firefox-2.0.0.6-source.tar.bz2;
      sha1 = "eb72f55e4a8bf08e8c6ef227c0ade3d068ba1082";
    };

  and the mirror list [http://nix.cs.uu.nl/dist/tarballs], we first
  try to download

    eb72f55e4a

  and if that fails, we use the original URL.

  The list of mirrors is not yet user-configurable.

* `fetchurl' now also accepts an argument `urls' instead of `url' for
  a list of alternative download locations, which fetchurl will try in
  sequence.

svn path=/nixpkgs/trunk/; revision=9190
2007-08-23 15:22:30 +00:00