Commit graph

930 commits

Author SHA1 Message Date
Robert Schütz
fa410ea633 python27: fix CVE-2021-23336
From the archive `python-gentoo-patches-2.7.18_p8.tar.xz` found at
https://dev.gentoo.org/~mgorny/dist/python/, I copied
`0024-3.6-bpo-42967-only-use-as-a-query-string-separator-G.patch`.
2021-04-03 15:10:01 +02:00
Frederik Rietdijk
9e83f91bb1 python39: 3.9.2 -> 3.9.3 2021-04-03 10:29:11 +02:00
Frederik Rietdijk
d754c20366 python38: 3.8.8 -> 3.8.9 2021-04-03 10:29:10 +02:00
Frederik Rietdijk
7a65bb76f1 pythonPackages: fix editable setuptools installations 2021-03-23 11:27:27 +01:00
Sandro Jäckel
b45c06fde8
cpython: allow $out reference again
This regression got introduced with 738746a34b
2021-03-15 03:20:32 +01:00
Frederik Rietdijk
0c1aa67215 python3Minimal and python3: don't override python3
This reduces the amount of packages that are required to use
builtins.fetchurl to fetch. Without this change, mime-types
would not be able to use fetchzip when mime-types support is
added to python3.
2021-03-14 17:01:44 +01:00
Frederik Rietdijk
738746a34b python3: express references to keep as a nix list 2021-03-14 17:00:59 +01:00
Frederik Rietdijk
5b1abe5dd2 python3: mimetypesSupport is optional 2021-03-14 15:45:15 +01:00
nicoo
faf0f8a8a4 cpython: Pull in the mime-types database
Closes #113901
2021-03-14 15:45:15 +01:00
Frederik Rietdijk
7617b03403 python39: hardcode path to tzdata
Python 3.9 has a new module `zoneinfo` which requires tzdata. By default
it searches TZPATH for folders containing `zoneinfo`.

This commit makes the dependency on tzdata pure.
2021-03-13 14:05:41 +01:00
Frederik Rietdijk
9d03ff5222 python: reproducible builds
Achieve reproducible builds of the interpreter. Note this meant
disabling optimizations again.
2021-03-13 13:11:50 +01:00
Frederik Rietdijk
277d488026 Merge staging into staging-next 2021-03-08 19:20:18 +01:00
Lassulus
4f85b3667b
Merge pull request #113142 from andersk/pypy-7.3.3
pypy, pypy3: 7.3.2 → 7.3.3
2021-03-07 16:52:42 +01:00
Ivan Babrou
b00c7c2d1d python37, python2: remove win64 workaround to fix aarch64-darwin
The issue manifests itself as the following on `aarch64-darwin`:

```
>>> import ctypes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/i8cq0xrjirz1rcp65wzcyhj6ypzlw9il-python3-3.7.10/lib/python3.7/ctypes/__init__.py", line 551, in <module>
    _reset_cache()
  File "/nix/store/i8cq0xrjirz1rcp65wzcyhj6ypzlw9il-python3-3.7.10/lib/python3.7/ctypes/__init__.py", line 273, in _reset_cache
    CFUNCTYPE(c_int)(lambda: None)
MemoryError
```

The commit we backport is included in Python 3.8, and it reverts
the change that was introduced all the way back in Python 2.7.
2021-03-03 16:02:07 -08:00
John Ericson
07ecf87693 treewide: Fix various tools wrappers "with packages"
Now that `buildEnv` is ready, always put `makeWrapper` in
`nativeBuildInputs`, rather than `buildInputs` or (worse) mucking around
with setup hooks by hand.

(C.f. #112276, which didn't catch these because the manual setup hook
sourcing is such a hack to being with!)

Fixes #114687
2021-03-02 22:38:04 +00:00
Frederik Rietdijk
0dc64d5d71 python: fix full builds by referring to the correct interpreter
The package set is an attribute of the interpreter. The function to
build an environment (`buildEnv`/`withPackages`) is part of the
interpreter. The interpreter is passed to itself, and needs to be
updated when overridden.

For cross-compilation we splice the package set, and for that the
various `build/host` interpreters and sets need to be available. We
select these currently through `pkgs.${pythonAttr}`. The `pythonAttr`
attribute was not fixed for `pythonFull`.

https://github.com/NixOS/rfcs/pull/83
https://github.com/NixOS/nixpkgs/pull/104201

We need a better solution for this because this is very brittle.
2021-02-27 11:12:05 +01:00
github-actions[bot]
5884dca2b9
Merge master into staging-next 2021-02-20 12:19:39 +00:00
Martin Weinelt
92087cb170 python37: 3.7.9 -> 3.7.10, fixup patches 2021-02-20 12:13:07 +01:00
Anders Kaseorg
d08ec2f195 pythonInterpreters.pypy36_prebuilt: Set pythonOnBuildForHost
This was broken by #105155.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-02-20 11:29:46 +01:00
Anders Kaseorg
1c5c184079 pythonInterpreters.pypy36_prebuilt: Add missing lib argument
This was broken by commit 001c0cbe54
(#110591).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-02-20 11:29:46 +01:00
Frederik Rietdijk
04f436940c python310: 3.10.0a4 -> 3.10.0a5 2021-02-20 10:03:11 +01:00
Frederik Rietdijk
8ab9914a8c python36: 3.6.12 -> 3.6.13 2021-02-20 10:03:11 +01:00
Martin Weinelt
85cde0d60f python27: Fix CVE-2021-3177
Thanks to the Gentoo team maintaining a fork of python2¹ we can easily
apply their backported patch for this security vulnerability.

[1] https://gitweb.gentoo.org/fork/cpython.git/
2021-02-20 10:03:11 +01:00
Frederik Rietdijk
2cbc212983 python37: 3.7.9 -> 3.7.10 2021-02-20 10:03:11 +01:00
Frederik Rietdijk
84edcadb4a python38: 3.8.7 -> 3.8.8 2021-02-20 10:03:11 +01:00
Frederik Rietdijk
a39b5b5c94 python39: 3.9.1 -> 3.9.2 2021-02-20 10:03:11 +01:00
Jan Tojnar
8f50f1ce10
Merge branch 'staging-next' into staging
Resolved the following conflicts:

- kernel flags between 09176d28a0 and 2b28822d8d
- clojure-lsp between 3fa00685ce and e03c068af5
2021-02-19 17:15:31 +01:00
Frederik Rietdijk
1e47264608 python.tests: test overriding functions 2021-02-19 10:12:21 +01:00
Martin Weinelt
d6d63aef7d
pytestCheckHook: add support for disabling arbitrary paths
Renames `disabledTestFiles` to the more genereric `disabledTestPaths` to
reflect that change.
2021-02-15 00:34:35 +01:00
Anders Kaseorg
4a8b6c0317 pypy, pypy3: 7.3.2 → 7.3.3
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-02-14 10:11:54 -08:00
github-actions[bot]
1c7975a26f
Merge master into staging-next 2021-02-13 18:15:59 +00:00
taku0
28adb5b0c1
Merge pull request #104150 from andersk/pypy-7.3.2
pypy, pypy3: 7.3.1 → 7.3.2
2021-02-13 21:48:05 +09:00
github-actions[bot]
257f90a43f
Merge master into staging-next 2021-02-13 12:19:38 +00:00
Frederik Rietdijk
b962afabd4 python.tests: add some comments to the tests 2021-02-13 09:44:51 +01:00
Martin Weinelt
0f5743bee2 pytestCheckHook: Fix support for multiple files.
Quoting here will lead to the shell treating them as a single item.
2021-01-24 15:44:45 -08:00
Martin Weinelt
45e5ebfad8 pytestCheckHook: Fix support for spaces in disabled file paths 2021-01-24 15:44:45 -08:00
Jan Tojnar
cc8fd11ffb
Merge branch 'master' into staging-next 2021-01-24 20:41:42 +01:00
volth
bc0d605cf1 treewide: fix double quoted strings in meta.description
Signed-off-by: Ben Siraphob <bensiraphob@gmail.com>
2021-01-24 19:56:59 +07:00
Jan Tojnar
01ee4ea574
Merge branch 'master' into staging-next 2021-01-24 00:09:45 +01:00
Ben Siraphob
001c0cbe54 pkgs/development/interpreters: stdenv.lib -> lib 2021-01-23 20:29:03 +07:00
github-actions[bot]
8103cb9089
Merge staging-next into staging 2021-01-23 01:18:03 +00:00
Peter Woodman
858eebe1f9 cpython: fix extensions when using a musl toolchain 2021-01-22 23:56:19 +01:00
github-actions[bot]
08c4f9056e
Merge staging-next into staging 2021-01-22 18:52:00 +00:00
Jonathan Ringer
0389d5b148 python310: 3.10.0a3 -> 3.10.0a4 2021-01-22 07:27:21 -08:00
Jan Tojnar
fc7bd322df
Merge branch 'staging-next' into staging 2021-01-19 13:50:04 +01:00
Jonathan Ringer
9bb3fccb5b treewide: pkgs.pkgconfig -> pkgs.pkg-config, move pkgconfig to alias.nix
continuation of #109595

pkgconfig was aliased in 2018, however, it remained in
all-packages.nix due to its wide usage. This cleans
up the remaining references to pkgs.pkgsconfig and
moves the entry to aliases.nix.

python3Packages.pkgconfig remained unchanged because
it's the canonical name of the upstream package
on pypi.
2021-01-19 01:16:25 -08:00
Sandro Jäckel
6f7f01a244 pytestCheckHook: Add disabledTestFiles option 2021-01-16 09:09:26 -08:00
Jan Tojnar
b813710c04
Merge branch 'master' into staging-next 2021-01-14 02:24:17 +01:00
Fabian Möller
07e7cea102 python36: enable ctypes.util.find_library patch 2021-01-13 13:13:02 +01:00
github-actions[bot]
bd9e17e21f
Merge staging-next into staging 2021-01-10 18:41:46 +00:00
github-actions[bot]
76394e4624
Merge master into staging-next 2021-01-10 18:41:43 +00:00
Profpatsch
2497553b23 toplevel: stdenv.lib -> lib
The library does not depend on stdenv, that `stdenv` exposes `lib` is
an artifact of the ancient origins of nixpkgs.
2021-01-10 17:44:59 +01:00
Frederik Rietdijk
7c61ecbefd Revert "Revert "python: 3.8.6 -> 3.8.7.""
This was originally reverted because of an invalid patch.
This has been fixed and now we want this update again.

This reverts commit e1ed9b3b08.
2021-01-09 13:12:44 +01:00
github-actions[bot]
7aced83bf9
Merge staging-next into staging 2021-01-08 12:29:39 +00:00
Orivej Desh
349585e778 python2: fix ctypes.util.find_library with gcc10
Fixes #108243
2021-01-08 11:19:39 +01:00
Frederik Rietdijk
5c8df2235a python3: don't use sysconfigdataHook on darwin
Resolves issue when building wheels which is a regression introduced by
1a65c5df5f

```
AssertionError: would build wheel with unsupported tag ('cp38', 'cp38',
'darwin_x86_64')
```
2021-01-08 09:34:36 +01:00
Jan Tojnar
0ea0dacc20
Merge branch 'staging-next' into staging 2021-01-07 13:06:41 +01:00
Jan Tojnar
f19eb635b4
Merge branch 'master' into staging-next
b04fc593e7 seems to have accidentally changed mkDerivation function for dfilemanager and solarus-quest-editor so I have reverted that here.
2021-01-07 13:04:31 +01:00
github-actions[bot]
2f7fa4bb88
Merge staging-next into staging 2021-01-05 00:57:02 +00:00
Jonathan Ringer
16174037b3 python38: remove obsolete find_library patch 2021-01-04 15:44:35 -08:00
Frederik Rietdijk
e1ed9b3b08 Revert "python: 3.8.6 -> 3.8.7."
Need to fix a patch first. Next iteration.

This reverts commit 507efdb11f.
2021-01-04 20:58:21 +01:00
Drew Hess
507efdb11f python: 3.8.6 -> 3.8.7. 2021-01-04 20:48:55 +01:00
John Ericson
5c2965145f treewide: Inline more of the static overlay
Picking up where #107238 left off. I think I'll have gotten all the easy
stuff with this.
2021-01-03 21:46:14 +00:00
Ben Wolsieffer
1a65c5df5f cpython: fix finding headers when cross-compiling extension modules 2020-12-28 13:09:00 +01:00
Lucas Ransan
188f1375d8 cpython: fix ctypes.util.find_library 2020-12-28 08:43:32 +01:00
github-actions[bot]
abb757ba28
Merge master into staging-next 2020-12-18 00:42:45 +00:00
Jonathan Ringer
222fafa02b python310: 3.10.0a2 -> 3.10.0a3 2020-12-17 11:18:47 -08:00
Greg Roodt
42d28c2a77 python: 3.9.0 -> 3.9.1 2020-12-12 19:32:08 +01:00
Frederik Rietdijk
76966f8ddd pythonPackages.pip-install-hook: remove --build option
Pip had the option --build to build in a custom or temporary directory.
Nowadays, pip just listens to TMPDIR, which we already set.

This option was deprecated and is removed in pip 20.3.
2020-12-01 14:44:26 +01:00
Frederik Rietdijk
9a63b3d3d6
Merge pull request #104781 from NixOS/staging-next
Staging next
2020-11-30 18:27:29 +01:00
adisbladis
49b8738a5f
python: Propagate packageOverrides to pythonForBuild
When overriding versions of build tools injected via hooks
`packageOverrides` was not taken into account and 2 incompatible
versions of the same package (`wheel` in this case) ended up in the
closure, causing the builds to fail.
2020-11-30 11:12:59 +01:00
Frederik Rietdijk
0d8491cb2b Merge master into staging-next 2020-11-29 13:51:10 +01:00
Frederik Rietdijk
455774e546 Python: add todo note regarding pythonForBuild 2020-11-28 17:36:23 +01:00
Frederik Rietdijk
5804c8a7f5 Python splicing: splice package sets, not Python derivation
In the original commit the various builds of Python were added to
`otherSplices`, instead of the intended Python package sets.
2020-11-28 17:36:23 +01:00
Frederik Rietdijk
6cf25f9dbd Python: rename parameters and arguments passed to passthru
As part of the splicing the build/host/target combinations of the interpreter
need to be passed around internally. The chosen names were not very clear,
implying they were package sets whereas actually there were derivations.
2020-11-28 17:36:23 +01:00
Frederik Rietdijk
8220b0449c python.tests: use self.callPackage instead of super.callPackage
super was incorrectly possible until https://github.com/NixOS/nixpkgs/pull/104201
got merged.
2020-11-28 16:57:07 +01:00
Frederik Rietdijk
cce2fd547b Python: use pythonPackagesBuildHost instead of pythonForBuild
Follow-up to #104201, related to #105113.
2020-11-28 16:36:03 +01:00
Jonathan Ringer
0a4575a1da python/hooks/pythonNamespaces: fix __pycache__ being empty, or not existing 2020-11-26 13:51:05 -08:00
Frederik Rietdijk
0a12b8a5e9 Revert "Revert "Revert "cpython: fix finding headers when cross-compiling extension modules"""
This still does not function without issues. E.g., bootstrapped-pip fails with Python 3.6 and 3.7
as well as 3.8 on 32-bit.

Because this is a stdenv-rebuild it needs to be tested significantly better

This reverts commit 6100bc29f7.
2020-11-25 10:34:23 +01:00
Jonathan Ringer
f5d9dd3050 python/hooks/pythonNamespaces: fix __pycache__ logic 2020-11-23 08:33:04 -08:00
Frederik Rietdijk
95d9ff16f3 python.tests: not when cross-compiling
Running the tests is not possible when cross-compiling.
2020-11-23 15:29:23 +01:00
Frederik Rietdijk
6100bc29f7 Revert "Revert "cpython: fix finding headers when cross-compiling extension modules""
Moving the sysconfig file caused spidermonkey_78 to fail to build. We now symlink it instead.

This reverts commit 6a23dde75b.
2020-11-23 15:29:23 +01:00
Frederik Rietdijk
7726f81b13 python.tests: use self.callPackage instead of super.callPackage
super was incorrectly possible until https://github.com/NixOS/nixpkgs/pull/104201
got merged.
2020-11-22 13:17:24 +01:00
Frederik Rietdijk
6a23dde75b Revert "cpython: fix finding headers when cross-compiling extension modules"
This breaks virtualenv https://github.com/NixOS/nixpkgs/issues/104483.
We should probably not move `_sysconfigdata` but just copy it for cross to
another place or output.

This reverts commit 11806b6ede.
2020-11-22 13:13:23 +01:00
Jonathan Ringer
8d210e2ea4 python3.pkgs.pipInstallHook: don't warn on script installation 2020-11-20 12:25:52 -08:00
Frederik Rietdijk
27d3a18ac7 buildPython*: don't catch conflicts when cross-compiling
Hack until we fix PYTHONPATH.
2020-11-19 20:59:16 +01:00
Ben Wolsieffer
11806b6ede cpython: fix finding headers when cross-compiling extension modules 2020-11-19 20:42:58 +01:00
Frederik Rietdijk
ea7b8978ef Merge master into staging-next 2020-11-19 20:08:15 +01:00
John Ericson
b57c5d4456 python: Use makeScopeWithSplicing
Now non-`buildInputs` that are python packages should be resolved
correctly.
2020-11-19 11:58:07 -05:00
John Ericson
330d468002 python: Use packages from previous stage in setup-hooks
This makes sense as these are tools we want to run at build time.
2020-11-19 11:58:03 -05:00
Frederik Rietdijk
da12fc6838 Merge staging-next into staging 2020-11-18 15:36:56 +01:00
Bernardo Meurer
7edf1ced82 pypy: enable aarch64-linux platform 2020-11-18 09:55:33 +01:00
Anders Kaseorg
9e09ad5182 pypy, pypy3: 7.3.1 → 7.3.2
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-11-18 00:21:41 -08:00
Anders Kaseorg
b7836e5ca6 pypy, pypy3: Follow Heptapod move for download URL
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-11-18 00:21:41 -08:00
Frederik Rietdijk
9fc4a4db5a Merge staging-next into staging 2020-11-13 20:59:08 +01:00
Frederik Rietdijk
3fceafce1e buildPython*: remove pythonRecompileBytecodeHook as dependency
This hook was added to get reproducible bytecode. Because it was causing
issues it was disabled, but still kept as a dependency. Now the main
issue with bytecode reproducibility has been resolved by updating pip to
20.2.4, we remove this hook as a dependency.

If a package with Python code is not yet reproducible, one could add
this hook to `nativeBuildInputs`.
2020-11-12 19:32:30 +01:00
Jonathan Ringer
5a5122418a python310: 3.10.0a1 -> 3.10.0a2 2020-11-11 23:13:44 -08:00
Frederik Rietdijk
20f001c01e Merge master into staging-next 2020-11-09 14:33:52 +01:00
Samuel Gräfenstein
22b73d1282 pkgs/*: add final newline to .nix files 2020-11-09 11:39:28 +00:00
Frederik Rietdijk
61a04f735a python3.pkgs.pip: 20.1.1 -> 20.2.4
Reproducible builds of pyproject projects using pip is resolved.

Fixes https://github.com/pypa/pip/issues/7808
Fixes https://github.com/NixOS/nixpkgs/issues/81441

The more recent c409f69480 caused trouble
with pyproject troubles and had to be reverted anyway.
https://github.com/NixOS/nixpkgs/pull/102222#issuecomment-722380794

Revert "pythonPackages.pip: make reproducible (#102222)"

This reverts commit c409f69480.

Revert "python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_INSTALL_TMPDIR"

This reverts commit aedbade43e.
2020-11-05 15:07:21 +01:00