The way in which Nixpks builds Ruby gems means that certain operations
by bundler *will not work*, namely `bundle install --redownload`.
According to the source the _cache/_ directory should have been kept,
however it seems through revisions to the file it has been purged.
Here was the comment from the original commit that introduced
buildRubyGem:
```
# Note:
# We really do need to keep the $out/${ruby.gemPath}/cache.
# This is very important in order for many parts of RubyGems/Bundler to not blow up.
# See https://github.com/bundler/bundler/issues/3327
```
Why is the _cache_ directory needed?
Bundler and RubyGems uses the cache as a source of truth.
When bundler executes `bundler install --redownload`, any gems it
discovers in the _GEM_PATH_ it assums must have their _.gem_ file
present in the cache (unaware it was installed from Nix).
Rather than downloading the gem from RubyGems the bundler code forcibly
re-installs the gem from the cache directory instead and **fails** if it
does not exist.
I've opened https://github.com/rubygems/rubygems/issues/4088 to see if
this failure should be soft and not so explicit; or fallback to fetching
the gem from scratch.
Without this change the following is the error:
```bash
> [nix-shell:~/code/nix/playground/jruby-bundler-rake]$ bundle install --force
[DEPRECATED] The `--force` option has been renamed to `--redownload`
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/nix/store/fis6nzrpw9pmcivr84qh5byfgm07qn10-jruby-9.2.13.0/lib/ruby/stdlib/jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Fetching gem metadata from https://rubygems.org/.
Using bundler 2.1.4
Installing hello-world 1.2.0
Bundler::GemNotFound: Could not find hello-world-1.2.0.gem for installation
An error occurred while installing hello-world (1.2.0), and Bundler
cannot continue.
Make sure that `gem install hello-world -v '1.2.0' --source
'https://rubygems.org/'` succeeds before bundling.
```
Wth the fix the following no woccurs:
```bash
[nix-shell:~/code/nix/playground/jruby-bundler-rake]$ bundle install --redownload
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/nix/store/69wjlj4yirp48rv1q03zxgd4xvf0150d-jruby-9.2.13.0/lib/ruby/stdlib/jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Fetching gem metadata from https://rubygems.org/.
Using bundler 2.1.4
Installing hello-world 1.2.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
```
```
[nix-shell:~/code/nix/playground/jruby-bundler-rake]$ ls -l /nix/store/cwl9n5073hqgpfhnw4wic13nrrgg9dn8-gem-env/lib/jruby/gems/2.5.0/cache/
total 8
lrwxrwxrwx 1 fmzakari primarygroup 102 Dec 31 1969 bundler-2.1.4.gem -> /nix/store/ifc8a0gsfkrhkv953rd4rz8bcspahi8y-bundler-2.1.4/lib/jruby/gems/2.5.0/cache/bundler-2.1.4.gem
lrwxrwxrwx 1 fmzakari primarygroup 110 Dec 31 1969 hello-world-1.2.0.gem -> /nix/store/xi9ln6n1mz2is5ppykjxqhhkpjq9zm6i-hello-world-1.2.0/lib/jruby/gems/2.5.0/cache/hello-world-1.2.0.gem
```
I have a minimal project that demonstrates this issue at https://github.com/fzakaria/jruby-bundler-nix-failure
Changed ruby version to 2.7.x to match upstream.
Added a gem config for gitlab-pg_query as it tries to download a source
tarball during the build process.
Also removed a patch for gitaly that has become obsolete by upstream fix
[here](de04077c25).
The way ruby loads gems and keeps track of their paths seems to not
always work very well when the gems are accessed through
symlinks. Ruby will then complain that the same files are loaded
multiple times; it relies on the file's full path to determine whether
the file is loaded or not.
This adds an option to simply copy all gem files into the environment
instead, which gets rid of this issue, but may instead result in major
file duplication.
According to https://endoflife.software/programming-languages/server-side-scripting/ruby
ruby 2.4 will go end-of-life in march, where the new release of nixpkgs
will be cut. We won't be able to support it for security updates.
Remove all references to ruby_2_4 and add ruby_2_7 instead where
missing.
Mark packages that depend on ruby 2.4 as broken:
* chefdk
* sonic-pi
Setting a Bundler version with GEM_PATH doesn't seem to work in Ruby
2.7, so we need to use the LOAD_PATH instead. Without this,
bundlerEnv environments will always use the version of Bundler that
comes with Ruby, which won't necessarily work because it isn't the
version that was used to generate the bundle.
For example, building ronn with Ruby 2.7 without this change results
in a broken executable, but it works (when built with all packaged
Ruby versions) after this change.
In building a gem whose native extension is a Rakefile, the previous
version of this code will call essentially `rake ""`, when it means to
call `rake`.
This change converts `""` into `[]` rather than `[""]`.
This fixes
nix-shell -p 'ruby.withPackages (const [])' ruby.devdoc
which otherwise wouldn't find documentation, unlike
nix-shell -p ruby ruby.devdoc
which would, because ruby has setup hooks to accomodate for this, that
were being masked by the withPackages wrapper.
Go beyond the obvious setup hooks now, with a bit of sed, with a skipped case:
- cc-wrapper's `dontlink`, because it already is handled.
Also, in nix files escaping was manually added.
EMP
Because the gemdir was referenced on the derivation, it would cause the
whole gemdir to get added to the store, which would in turn force the
derivation to be rebuilt whenever unrelated folder files would change.
