Initial support for installing default gems. #566
Conversation
The logic here is pulled from MRI's rbinstall.rb default gem logic and the actual logic for installing gems. The combined logic works as follows: * The normal dependency chain is followed, unlike the rbinstall logic. * The gem's full specification (with complete file list) is dumped to GEM_HOME/specifications/default/GEM_NAME.gemspec. This covers the bulk of the rbinstall logic. * Only the bin files from the gem are extracted to the normal gem install location. This allows a stub binscript to boot from a combination of gem's bin and stdlib's libraries, as in the rbinstall logic. Remaining work to do: * Unpack gem lib and ext contents into appropriate location for the stdlib. This will overwrite existing files, and is generally intended to do a dev-time update of a given default gem's contents in stdlib. * Install a binstub that works with both stdlib (default gem) libraries and installed gems, as in Ruby 2.0's rake, rdoc, ri commands. Primarily, this binstub needs to fallback to just a simple require when gem loading fails.
|
Worth pointing out: rbinstall.rb does not unpack gem contents into stdlib nor does it create the default-aware binstubs. I mentioned that as work to-do in my original PR mostly because it seems logical to have a way to pull down a gem and replace local ext/lib contents with updated gem contents. It would be bonus work to implement this. |
There was a problem hiding this comment.
:default seems to ambiguous. How about :install_as_default?
* Use :install_as_default to be less ambiguous in options * Add logic to ensure specification/default dir is created
|
I realize now there's a "flaw" in this PR that actually should probably be fixed in RubyGems. Default gem logic works by reading in specs from specifications/default and using the file list as triggers to attempt to boot those gems. However, the list of files it expects needs to be the require names rather than the file list normally provided by gems. My PR uses the actual specification files from the gem, so it includes files not normally requirable (README, bin/, etc), and includes prefixes on those that are requirable (lib/, ext/*). I believe RG default gem support needs to be modified to be able to use a proper, ummodified gemspec (as in my PR) and to build its list of require file hooks based on spec.files and spec/require_paths. The logic would be:
Comments welcome...I may try to implement this, but would welcome help. I'm also not sure it can be done as a non-breaking change since MRI 2.0 has shipped with the old logic and the old modified gemspec files with stripped paths. |
MRI 2.0 shipped with its own default spec generator that rewrites the gemspec file to contain a list of bare require names in spec.files. This required extra logic in rbinstall.rb and makes it impossible to simply copy a gemspec from a gem into the default directory to register that gem as a default gem. PR ruby#566 adds --default functionality to `gem install` that copies the gem's unmodified gemspec to the default directory, but since unmodified gemspec contains a list of gem-relative filenames, it could not use the old default specification logic. This commit modifies default spec registration to support both formats. In the "old" style, specifications are registered with their raw filenames; we detect old style by checking the first file to see if it is prefixed with any spec.require_paths. In the "new" style, spec.require_paths are stripped off the spec's filenames before registration, and only files that start with a require_path are registered. Both the old format and the new format continue to work with this change, and we can migrate toward default specs being the unmodified originals rather than custom-built as in MRI 2.0.
|
I have added a commit that modifies Gem.register_default_spec to support both the old and new gemspec formats. You can see the commit entry for more details, but in short... old style (MRI 2.0 custom-build specs) register as they do now, using all entries from spec.files; new style (unmodified gemspec) works by stripping require paths from files and only registering requirable names. |
|
Regarding the flaw, master won't be merged to Ruby trunk as it has new features that should appear in Ruby 2.1. |
|
I'm hoping we can include this functionality in JRuby 1.7.5, so either the PR can be modified to work on RG 2.0 or hopefully 2.x will be released before then. Backporting my changes won't be difficult. |
|
I'm confused...I do not see a 2.0 branch in git. Is there a way I can rework this patch to get into a RubyGems 2.0 release, or will we have to wait until 2.1 (and if the latter, how long will that be?). |
|
If we need to do another 2.0.x release I'll start from 2.0.3 for the branch. What is your JRuby 1.7.5 timeframe? I hope to have a 2.1 release in the next three to four week timeframe. Currently there are 22 open issues on the 2.1 milestone and I've been working on those (I suspect a few more fixes/features will show up in there). I've been working the issues from oldest to newest and this is currently third-newest. If JRuby 1.7.5 is imminent I can apply and backport this more quickly. |
|
I'd need to confer with @enebo but I don't think 1.7.5 is imminent. We're probably looking at a release in late July at the earliest, and before JRubyConfEU (mid-August) at the latest. I'd really like to get MRI 2.0-style default gem support into that release...so let me know if there's anything I can do to help make that happen. |
Initial support for installing default gems.
|
BTW, the flaw workaround logic will probably have to live for a long time (forever) since all installed MRI 2.0 have the old default specification format created by rbinstall.rb. |
|
Yep, but that's fine, it'll be just like the 1.8.7 and 1.9.1 baggage we're still hauling around. |
2 participants
The logic here is pulled from MRI's rbinstall.rb default gem logic
and the actual logic for installing gems. The combined logic works
as follows:
logic.
to GEM_HOME/specifications/default/GEM_NAME.gemspec. This covers
the bulk of the rbinstall logic.
install location. This allows a stub binscript to boot from a
combination of gem's bin and stdlib's libraries, as in the
rbinstall logic.
Remaining work to do:
the stdlib. This will overwrite existing files, and is generally
intended to do a dev-time update of a given default gem's
contents in stdlib.
libraries and installed gems, as in Ruby 2.0's rake, rdoc, ri
commands. Primarily, this binstub needs to fallback to just a
simple require when gem loading fails.