Updating Ruby 3.0.0 to 3.0.7

[mudge]

I'm currently unable to compile a Musl gem for re2 on Ruby 3.0.0 as it has a bug when compiling C++ extensions that was fixed in Ruby 3.0.5.

What is the exact version of Ruby used when 3.0.0 is included in RUBY_CC_VERSIONS? If it is less than 3.0.5, is there any chance of getting it updated?

[flavorjones]

@mudge "3.0.0" means the source tarball from https://cache.ruby-lang.org/pub/ruby/3.0/ruby-3.0.0.tar.gz (it could mean a prerelease but in this case does not).

I was thinking about how much it sucks that rake-compiler-dock exposes the full rake-compiler version tag, and that gems need to pin their cross-compilation to specific versions of Ruby. That's the only reason not to do this, but maybe that's just part of the upgrade process for gem owners.

Let me see how hard it is to bump all the rubies to the latest patch for each minor.

[mudge]

Rather than prescribing a solution, the exact problem I'm running into is the following error during a call to try_compile in re2's extconf.rb when compiling for x86_64-linux-musl (as extracted from the resulting mkmf.log):

In file included from /usr/local/rake-compiler/ruby/x86_64-unknown-linux-musl/ruby-3.0.0/include/ruby-3.0.0/ruby/ruby.h:39,
                 from /usr/local/rake-compiler/ruby/x86_64-unknown-linux-musl/ruby-3.0.0/include/ruby-3.0.0/ruby.h:38,
                 from conftest.c:1:
/usr/local/rake-compiler/ruby/x86_64-unknown-linux-musl/ruby-3.0.0/include/ruby-3.0.0/ruby/internal/memory.h:275:16: error: conflicting declaration of 'void* ruby_nonempty_memcpy(void*, const void*, size_t)' with 'C' linkage
  275 | #define memcpy ruby_nonempty_memcpy
      |                ^~~~~~~~~~~~~~~~~~~~
/usr/local/rake-compiler/ruby/x86_64-unknown-linux-musl/ruby-3.0.0/include/ruby-3.0.0/ruby/internal/memory.h:265:1: note: previous declaration with 'C++' linkage
  265 | ruby_nonempty_memcpy(void *dest, const void *src, size_t n)
      | ^~~~~~~~~~~~~~~~~~~~
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <re2/re2.h>
4: int main() { return 0; }
/* end */

This error matches the one reported in Ruby bug #17788.

I've pushed a test repo to reproduce this error: https://github.com/mudge/rcd_test_cpp Running bundle exec rake should be enough to trigger the error.

[mudge]

To give some confidence that the fix to Ruby bug #17788 will resolve this, I've tested cross-compiling the gem with ruby/internal/memory.h patched (see https://github.com/mudge/rcd_test_cpp/pull/1/files) and that seems to do the trick.

[flavorjones]

@mudge Thanks for doing more work here to isolate the problem and the patch. Unfortunately it's not easy to patch the rubies being built by rake-compiler.

I'm going to spend some time this weekend trying to isolate the issue with the 3.0.x series, mostly because I've learned that issues like this tend to crop up again in a different form if I fail to understand them when I run into them.

But if I can't figure that out this weekend, we'll fall back to patching the 3.0 series for you (which, although it may be a bit of work, is low-risk work).

[mudge]

Thanks, @flavorjones and happy new year.

There’s no particular rush from my end as the current version of re2 should compile fine on 3.4 and there’s a release candidate should people need a precompiled, native version (it’s just that I don’t want to promote that to the latest public version as it doesn’t need to drop support for old versions of Ruby).

[flavorjones]

@mudge Question for you: I thought you had decided to drop support for precompiled Rubies < 3.1? That's what the re2 changelog seems to say, so I'm just curious if you changed your mind and if so, what changed it?

Also some progress: the "symbol not found" error seems to have been introduced in 3.0.5 (because 3.0.4 went green in #136), and that's a pretty small changeset.

[mudge]

I changed my mind. Initially, I thought it would be too much effort to support older versions of Ruby given my initial GitHub Actions runs with the upgraded rake-compiler-dock and separate gnu and musl gems. However, once I installed the latest versions of RubyGems for 2.6, 2.7 and 3.0, the gem continued to compile, install, and work fine and it felt excessive for me to drop three versions of Ruby in one go (I also have the unpopular opinion that I should make the gem available to as many older versions as is reasonable).

As for 3.0.4 and 3.0.5, the diff seems to contain Darwin specific linker changes re bundle loader that might be worth investigating?

[flavorjones]

looks around, sweeps arms grandly at the universe

This is what "too much effort to support older versions of Ruby" looks like, IMHO, and is why we're having a conversation at #77 about dropping older rubies.

[flavorjones]

OK, I'm 90% sure the issue with both v2.7.7 and v3.0.5 is the same reason we can't use any of the 3.1 series above 3.1.2: ruby/ruby#6193 originally in 3.2.0-preview2 but also backported to 3.1.2, 3.0.5 and 2.7.7.

I actually fixed this in the 3.2 series here and here! But those fixes weren't backported to 3.1, 3.0, or 2.7 because we have always just used the .0 release in rake-compiler-dock and so I didn't know it was a problem.

I'll see if I can work around it.

[flavorjones]

Ah, ok, so my diagnosis wasn't exactly right, but it's absolutely related to ruby/ruby#6193 introducing -bundle_loader. Applying the -flat_namespace fix we used for Ruby 3.2 to those other rubies seems to work.

And fixing the 3.1.x problem described in https://bugs.ruby-lang.org/issues/19239 was a matter of patching rake-compiler to explicitly build miniruby, allowing us to bump to 3.1.6 as well.

[flavorjones]

Closed by #135. Will cut a release shortly.