Commit graph

380 commits

Author SHA1 Message Date
Silvan Mosberger
cf47b9a5c0
Merge pull request #270537 from 9999years/packagesFromDirectory
lib.packagesFromDirectoryRecursive: init
2023-12-19 22:03:01 +01:00
Rebecca Turner
090b929b8a
lib.packagesFromDirectoryRecursive: init
Co-authored-by: Gabriella Gonzalez <GenuineGabriella@gmail.com>
2023-12-19 09:48:17 -08:00
Rebecca Turner
fa9727cf1e
lib: modules.sh should check JSON output for predictability
Currently, the `lib/tests/modules.sh` test checks the output of
`nix-instantiate --eval` without `--json`, which outputs an unspecified
human-readable format.

This patch modifies `modules.sh` to use the `--json` output instead, to
be robust against future changes to `nix-instantiate` output.
2023-12-19 09:23:09 -08:00
Adam Joseph
8f34a10d6a lib/tests/release.nix: temporary reference to pkgs/test/release
This commit temporarily adds pkgs/test/release to the
lib/tests/release.nix test suite, because ofborg already knows about
that entry point.

We should move the list of test entry points out of ofborg and into
a central place in nixpkgs:

  https://github.com/NixOS/nixpkgs/issues/272591

Once we do that we won't need to have this ugly kludge in an
inappropriate place.
2023-12-15 05:13:50 -08:00
Robert Hensing
6c8fb49bfc
Merge pull request #273004 from hercules-ci/attrset-path-longest-prefix
lib.attrsets.longestValidPathPrefix: init
2023-12-11 13:25:45 +01:00
Maximilian Bosch
d56f942e43
Merge pull request #272709 from hercules-ci/module-system-test-pr-131205
lib/modules: Test optionless module errors from #131205
2023-12-10 13:49:02 +01:00
Robert Hensing
584463c744
Merge pull request #272764 from tweag/anyBool
lib.types.anyBool: init
2023-12-10 06:03:50 +01:00
Robert Hensing
83712164e6 lib/modules: Clarify test assertions 2023-12-09 20:38:29 +01:00
Robert Hensing
afb1a2e376 lib/modules: Test optionless module errors from #131205 2023-12-09 14:15:49 +01:00
Robert Hensing
7d993b9521 lib.attrsets.hasAttrByPath: Document law and laziness, and test it 2023-12-08 23:19:09 +01:00
Robert Hensing
72bd4bbb58 lib.attrsets.longestValidPathPrefix: init
Allows finding the most specific path that exists.
This is useful for error messages relating to attribute paths.
2023-12-08 23:15:13 +01:00
Robert Hensing
67cc78643d lib.sortOn: init
A more efficient sort in some cases, and often convenient.

This exposes `lib.lists.sortOn` immediately on `lib`, because it is
a sibling of `sort`, which is already present there.
Omitting it would lead to more confusion, and worse outcomes.
There's no confusion about the types `sort` or `sortOn` operate on.

Haskell agrees about the type for `sortOn`, and it is in its `base`.
2023-12-08 22:15:29 +01:00
Silvan Mosberger
8d3978c149 lib.types.boolByOr: init
This type is necessary to have correct merging behavior for
`allowUnfreePredicate` and `allowInsecurePredicate`

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-08 21:50:29 +01:00
Robert Hensing
51357572f2
Merge pull request #269552 from adisbladis/lib-matchattrs-list-allocs
lib.attrsets.matchAttrs: Avoid some list allocations when walking structure
2023-11-27 14:44:37 +01:00
adisbladis
013a0a1357 lib.attrsets.matchAttrs: Avoid some list allocations when walking structure
Benchmarks (`nix-instantiate ./. -A python3`):

- Before:
``` json
{
  "cpuTime": 0.29049500823020935,
  "envs": {
    "bytes": 4484216,
    "elements": 221443,
    "number": 169542
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 53086800
  },
  "list": {
    "bytes": 749424,
    "concats": 4242,
    "elements": 93678
  },
  "nrAvoided": 253991,
  "nrFunctionCalls": 149848,
  "nrLookups": 49612,
  "nrOpUpdateValuesCopied": 1587837,
  "nrOpUpdates": 10104,
  "nrPrimOpCalls": 130356,
  "nrThunks": 358981,
  "sets": {
    "bytes": 30423600,
    "elements": 1859999,
    "number": 41476
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236145,
    "number": 24453
  },
  "values": {
    "bytes": 10502520,
    "number": 437605
  }
}
```

- After:
``` json
{
  "cpuTime": 0.2946169972419739,
  "envs": {
    "bytes": 3315224,
    "elements": 172735,
    "number": 120834
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 48718432
  },
  "list": {
    "bytes": 347568,
    "concats": 4242,
    "elements": 43446
  },
  "nrAvoided": 173252,
  "nrFunctionCalls": 101140,
  "nrLookups": 73595,
  "nrOpUpdateValuesCopied": 1587837,
  "nrOpUpdates": 10104,
  "nrPrimOpCalls": 83067,
  "nrThunks": 304216,
  "sets": {
    "bytes": 29704096,
    "elements": 1831673,
    "number": 24833
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236145,
    "number": 24453
  },
  "values": {
    "bytes": 8961552,
    "number": 373398
  }
}
```
2023-11-27 11:20:50 +13:00
adisbladis
4b4d413817 lib.meta: Avoid attrset allocation in platformMatch
Benchmarks (`nix-instantiate ./. -A python3`)

- Before
``` json
{
  "cpuTime": 0.30625399947166443,
  "envs": {
    "bytes": 4484216,
    "elements": 221443,
    "number": 169542
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 53091024
  },
  "list": {
    "bytes": 749424,
    "concats": 4242,
    "elements": 93678
  },
  "nrAvoided": 253991,
  "nrFunctionCalls": 149848,
  "nrLookups": 49614,
  "nrOpUpdateValuesCopied": 1588326,
  "nrOpUpdates": 10106,
  "nrPrimOpCalls": 130356,
  "nrThunks": 359013,
  "sets": {
    "bytes": 30432320,
    "elements": 1860540,
    "number": 41480
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236218,
    "number": 24459
  },
  "values": {
    "bytes": 10504632,
    "number": 437693
  }
}
```

- After
```
{
  "cpuTime": 0.29695799946784973,
  "envs": {
    "bytes": 3296712,
    "elements": 169055,
    "number": 121517
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 49044992
  },
  "list": {
    "bytes": 504928,
    "concats": 4242,
    "elements": 63116
  },
  "nrAvoided": 175403,
  "nrFunctionCalls": 110554,
  "nrLookups": 44907,
  "nrOpUpdateValuesCopied": 1588326,
  "nrOpUpdates": 10106,
  "nrPrimOpCalls": 82330,
  "nrThunks": 306625,
  "sets": {
    "bytes": 29943328,
    "elements": 1843076,
    "number": 28382
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 236218,
    "number": 24459
  },
  "values": {
    "bytes": 9037752,
    "number": 376573
  }
}
```
2023-11-25 11:05:23 +13:00
Silvan Mosberger
ff562fa5fc
Merge pull request #266443 from amjoseph-nixpkgs/pr/lib-tests-no-aws-sdk 2023-11-19 06:17:18 +01:00
Adam Joseph
54c9a08aaf lib.tests: build nix without flaky aws-sdk-cpp
The aws-sdk-cpp tests are flaky.

Since pull requests to staging cause nix to be rebuilt, this means
that staging PRs end up getting false CI failures due to whatever is
flaky in the AWS SDK tests.  Since none of our CI needs to (or
should be able to) contact AWS S3, let's just omit it all.  Bonus:
the tests build way faster.
2023-11-18 20:19:10 -08:00
Silvan Mosberger
2dfb1d36cf lib.fileset.gitTracked/gitTrackedWith: init
A configuration parameter for gitTrackedWith will be introduced in the
next commit
2023-11-16 01:12:15 +01:00
Felix Buehler
66261e9961 lib.lists.allUnique: init 2023-11-14 19:52:32 +01:00
Silvan Mosberger
17012aa0d2
Merge pull request #261676 from h7x4/lib-add-replicatestring
lib.strings: add `replicate`
2023-11-07 18:29:31 +01:00
Silvan Mosberger
add254658a lib.filesystem: Don't test Nix-specific error messages
In https://github.com/NixOS/nix/pull/9269 the error messages change
which would've broken this test.
2023-11-01 18:56:03 +01:00
h7x4
206d20426c
lib.strings: add replicate
`strings.replicate` returns n copies of a string, concatenated into a new
string

Co-authored-by: Silvan Mosberger <github@infinisil.com>
2023-10-31 20:25:41 +01:00
Silvan Mosberger
e55ee23bf6
Merge pull request #260971 from Gerg-L/getExe
lib.getExe': check arguments
2023-10-31 17:29:15 +01:00
Gerg-L
d2161d0632
lib/tests: add tests for getExe' and getExe 2023-10-16 20:29:28 -04:00
Silvan Mosberger
302ec1a3cf
Merge pull request #258866 from schuelermine/mkPackageOptionV4
lib/options: mkPackageOption: document better, add pkgsText and usePname options, refactor
2023-10-16 20:40:48 +02:00
Robert Hensing
0a10279342 lib.makeOverridable: fix functionArgs on returned function 2023-10-11 22:56:53 +02:00
Silvan Mosberger
5323fbf703
Merge pull request #254452 from flyingcircusio/lib-attrsToList
lib.attrsets.attrsToList: add function
2023-10-10 19:49:17 +02:00
Oliver Schmidt
d70633f91c lib.attrsets.attrsToList: add function
For transforming back between lists and attrsets, it makes sense to have
a quasi-inverse of `builtins.listToAttrs` available as a library
function.

Co-authored-by: Silvan Mosberger <github@infinisil.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-10-10 16:11:01 +02:00
Anselm Schüler
55ab538abf
lib/tests: add test for pkgsText parameter and package set selection of mkPackageOption 2023-10-10 13:13:38 +02:00
Anselm Schüler
d27d38c13a
lib/tests: add more tests for mkPackageOption 2023-10-06 14:08:46 +02:00
linsui
a50cea84be lib.gvariant: make the tests really work 2023-09-28 16:30:05 +02:00
Silvan Mosberger
dd72ff27f7 lib.attrsets.foldlAttrs: Make stricter
See the parent commit for the same change to lib.lists.foldl'
2023-09-27 02:43:59 +02:00
Silvan Mosberger
3b6169f87b lib.lists.foldl': Make strict in the initial accumulator
To maintain backwards compatibility, this can't be changed in the Nix language.
We can however ensure that the version Nixpkgs has the more intuitive behavior.
2023-09-27 02:43:59 +02:00
Silvan Mosberger
9893fee947 lib.lists.foldl': Add tests 2023-09-26 18:55:54 +02:00
Robert Hensing
00e5487906
Merge pull request #249243 from lf-/jade/declarationsWithLocations
nixos/modules: Add declarationPositions
2023-09-17 19:43:07 +02:00
Robert Helgesson
54bc9b45a3
Merge pull request #250220
lib.generators.toGitINI: escape string values in configuration
2023-09-11 23:44:42 +02:00
Jade Lovelace
a1d3882307 nixos/modules: Add declarationPositions
What it does: line and column level *declaration* position information:

$ nix repl .
nix-repl> :p nixosConfigurations.micro.options.environment.systemPackages.declarationPositions
[ { column = 7; file = "/nix/store/24aj3k7fgqv3ly7qkbf98qvphasrw9nb-source/nixos/modules/config/system-path.nix"; line = 63; } ]

Use cases:
- ctags over NixOS options, as will be presented at NixCon 2023 ;)
- improving the documentation pages to go to the exact line of the
  declarations.

Related work:
- https://github.com/NixOS/nixpkgs/pull/65024

  This one does it for all *definitions* rather than declarations, and
  it was not followed through with due to performance worries.
- https://github.com/NixOS/nixpkgs/pull/208173

  The basis for this change. This change is just a rebase of that one.
  I split it out to add the capability before adding users of it, in
  order to simplify review. However, the ctags script in there is a
  sample user of this feature.

Benchmarks: conducted by evaluating my own reasonably complex NixOS
configuration with the command:
`hyperfine -S none -w 1 -- "nix eval .#nixosConfigurations.snowflake.config.system.build.toplevel.outPath"`

```
Benchmark 1: nix eval .#nixosConfigurations.snowflake.config.system.build.toplevel.outPath
  Time (mean ± σ):      8.971 s ±  0.254 s    [User: 5.872 s, System: 1.388 s]
  Range (min … max):    8.574 s …  9.327 s    10 runs

Benchmark 1: nix eval .#nixosConfigurations.snowflake.config.system.build.toplevel.outPath
  Time (mean ± σ):      8.766 s ±  0.160 s    [User: 5.873 s, System: 1.346 s]
  Range (min … max):    8.496 s …  9.033 s    10 runs
```

Summary of results: it seems to be in the noise, this does not cause any
visible regression in times.
2023-09-08 11:48:40 +02:00
Silvan Mosberger
d66929b7b1
Merge pull request #245623 from tweag/fileset.toSource
File set combinators base: `lib.fileset.toSource`
2023-09-02 04:07:02 +02:00
Silvan Mosberger
465e05c0c5 lib.fileset.toSource: init 2023-09-01 15:46:04 +02:00
Jan Tojnar
434d160d7c
Merge pull request #234615 from linsui/dconf
nixos/dconf: support generating from attrs
2023-08-28 18:31:52 +02:00
VwCSXg
147560180c lib.generators.toGitINI: added test
Added basic generators.toGitINI test.
Mostly taken from 958c06303f/tests/modules/programs/git/git.nix.
The ${"\t} escape is used so that the lines aren't recognized as "Wrong
indent style".
2023-08-21 13:12:40 +02:00
Robert Hensing
5ad01f1b6f lib/modules: Report a better error when option tree has bare type
Improves on 0d472a6201
- https://github.com/NixOS/nixpkgs/pull/242339

We actually do have the file name.

Thanks Shawn8901 for the [feedback]!

feedback: https://github.com/NixOS/nixpkgs/pull/242339#issuecomment-1683107055
2023-08-18 11:36:51 +02:00
linsui
da614d98e9 lib/gvariant: init 2023-08-15 19:20:39 +08:00
Silvan Mosberger
8fab18d4c1 lib.removePrefix: Add tests 2023-08-14 23:29:00 +02:00
Silvan Mosberger
87c5a6a84f
Merge pull request #243511 from tweag/lib.lists.hasPrefix
`lib.lists.{hasPrefix,removePrefix}`: init
2023-08-14 21:15:54 +02:00
Robert Hensing
0d472a6201 lib/modules: Report a good error when option tree has bare type
Note that this removes the possibility of declaring an option
named `_type`.
2023-08-14 10:44:56 +02:00
Robert Hensing
53b289836b
Merge pull request #243139 from hercules-ci/modules-test-default-argument
lib/tests/modules: Test that _module.args works when a default argume…
2023-08-02 20:12:37 +02:00
Robert Hensing
19f1d7da06
Merge pull request #245271 from sternenseemann/module-system-merge-no-type
lib/modules: handle typeless options in mergeModules
2023-07-27 10:55:08 +02:00
sternenseemann
9c35f44999 lib/modules: handle typeless options in mergeModules
mkOption does not require a `type` argument and does not set the
resulting attribute if it is not given. Consequently, we need to be
prepared to merge options that have no type information.
2023-07-26 23:58:04 +02:00