Add 3.4-asan builds

[ivoanjo]

What does this PR do?

This PR introduces a new "3.4-asan" build, based on the existing asan builds, but just pointed at whatever's the latest tag in the 3.4 series.

Motivation:

The intention of "3.4-stable" is to provide the latest up-to-date stable Ruby, so that we can reliably use it as a breaking CI step.

As discussed in ruby/setup-ruby#682, the current ruby-asan builds are a bit of a "sharp edge" when used in CI because they may break due to changes that are completely unrelated to asan.

Building asan rubies is a bit awkward still, as e.g. ruby-build and other version managers don't have support for it, and it requires very modern versions of specific system tools (e.g. clang).

Additional Notes:

After some back-and-forth, the changes are reasonably minimal.

In particular, I decided to not touch the logic that determines weather there's a more recent commit to build or not. This does mean that if ruby master sees no commits, but there's a new 3.4 stable release, this won't be picked up immediately; and it also means that if there's a new master commit and no change to the 3.4 branch we still rebuild 3.4-asan.

My thinking is that the extra complexity to individually take care of the caching for both branches is not worth the trouble vs doing some extra rebuilds for 3.4-asan.

Let me know if you're not convinced, and I can change that.

How to test the change?

I've built this in the downstream fork, and manually downloaded the resulting Ruby and it seems to be in good shape and with asan working fine.

• Successful run: https://github.com/DataDog/ruby-dev-builder/actions/runs/12827351740
• Resulting builds: https://github.com/DataDog/ruby-dev-builder/releases/tag/v20250117.103455

[eregon]

I'm finally taking a look at this.

It seems quite wasteful to me to rebuild 3.4-asan for every ruby/ruby master commit.

Also the existing asan is already the slowest job by a wide margin: example and in your run.
IMO 20min is already too long for a -dev-builder and that notably caused extra delays when adding ubuntu-*-arm support.
We can see also in Total duration 24m 41s vs 21m 51s , but maybe that's just noise.

So I'm thinking maybe debug and asan builds should be daily or even less frequent jobs maybe weekly, separate from the head builds so they don't slow those.
I'm not quite sure how to achieve that though.

[eregon]

There is a ruby_3_4 branch which could simplify quite a bit of logic added here.
It wouldn't necessarily be a release, but at least that branch should equivalent to a release + potentially some extra bug fixes.

[ivoanjo]

There is a ruby_3_4 branch which could simplify quite a bit of logic added here.
It wouldn't necessarily be a release, but at least that branch should equivalent to a release + potentially some extra bug fixes.

Actually that's even easier and in fact I started there. Then I decided to get a bit fancy haha. My assumption is that using the latest released snapshot is a bit more stable (and going back to your point above, it should mean even less rebuilds overall), but I'm happy to revert this back to directling using ruby_3_4.

[eregon]

Could you do this change to see how much it simplifies?
It's probably good to keep the current diff in some other branch as those semantics might be desirable, but I'm keen to simplify if we can.

[ivoanjo]

Also the existing asan is already the slowest job by a wide margin: example and in your run.
IMO 20min is already too long for a -dev-builder and that notably caused extra delays when adding ubuntu-*-arm support.
We can see also in Total duration 24m 41s vs 21m 51s , but maybe that's just noise.

Do you know what kind of account does the Ruby org have on GitHub? I wonder if larger runners are an option to make this a bit better.

So I'm thinking maybe debug and asan builds should be daily or even less frequent jobs maybe weekly, separate from the head builds so they don't slow those.
I'm not quite sure how to achieve that though.

I was checking the docs and it looks like it's possible to have multiple schedules and to distinguish between them.

What I'm not entirely sure is -- is it ok for some releases to not have the asan variants as far as setup-ruby downstream goes? Or would that make life harder on that side?

[eregon]

Re larger runners I think they are all paid.
But apparently the new ubuntu-*-arm runners are faster so maybe that could be worth a try.

I was checking the docs and it looks like it's possible to have multiple schedules and to distinguish between them.

Interesting, what I would be really interested is to skip debug & *asan on workflow_dispatch (when triggering manually).

What I'm not entirely sure is -- is it ok for some releases to not have the asan variants as far as setup-ruby downstream goes? Or would that make life harder on that side?

Yeah that's the tricky part. Right now setup-ruby always use the latest release and expects all files to be there.
I see two solutions:

• Create another repo for debug & asan builds, and this repo would only build head. Would need to ask hsbt if that's OK. But that would mean having two workflow files. It's possible to share with a composite action but somewhat tricky and more complicated.
• Use a separate fixed release for debug & asan builds. Then whenever making a new build, delete the old assert and upload the new build (there is no way on GH to replace a release asset atomically unfortunately). That means at least in theory there would be a time where the asset is missing, but from others doing that it seems little problem in practice (maybe caching accidentally helps, or it's just fast enough to be very rare).
  It would mean a single debug/asan build available, and they might not be from the same commit (if one fails), but that might be OK. One big advantage is if asan or debug fails (happened a couple times in the past) at least we don't block head builds.
• (we could also store the latest build somehow in some file in the repo, but IIRC that works poorly as e.g. there is some caching on raw files and I would like to keep avoiding querying URLs in setup-ruby)

The 2nd seems easier and a smaller change.
Is it something you could try?

TBH I am on the fence to just merge this as-is, but also I'm confident in some future I would regret it and maybe it's just a good occasion to make debug/asan not slow down and not block head builds (relevant whenever there is a new runner).

[ivoanjo]

Re larger runners I think they are all paid.

Yeah, my thinking is -- maybe the Ruby org had some kind of nice-OSS-account that afforded that use, given its importance for GitHub or something ;D

Interesting, what I would be really interested is to skip debug & *asan on workflow_dispatch (when triggering manually).

Stared at the docs for a while, and seems to be possible :)

Is it something you could try?

Yea, I can take a stab at it.

I'm with you -- I think it seems really awkward to create additional releases, repos, etc.

Replacing seems doable as you said but... it seems like a sharp edge -- e.g. it seems annoying that I ran into some issue (maybe I even got a core dump) and I would like to take a look at the Ruby that ran into it, and... it's already gone. Having the archive of built binaries seems to be a useful property?

Maybe we could try it in a slightly different direction. What if we introduced reuse of builds?

That is, the latest release would always have a full set of builds BUT some of them (debug/asan) might be reused from the previous run.

This would have a few interesting properties:

• We'd avoid doing the 3.4-asan build when the last commit was not changed (which it's quite likely to happen)
• We could reuse the latest debug build that was successful if debug is broken
• We could have asan/debug on a different schedule entirely. E.g. maybe they only build once per week, and then every other build in between just copies them over.
• Based on the separate schedule, if debug was broken, then the more-regular builds wouldn't be affected, only whatever builds we put together with debug (probably asan, and that seems fine)

And this is how I see it fitting within the current flow:

• There would be a download/reupload step that runs when asan/debug don't
• We'd change the comment on the release to include a bit more details (e.g. to make it clear when builds were reused or not, so as to not cause confusion)
• Add the additional schedule for these builds
• Have a workflow_dispatch option that takes as input if it should build all or reuse

...something along these lines?

It seems doable within GHA with a few lines each.

TBH I am on the fence to just merge this as-is, but also I'm confident in some future I would regret it and maybe it's just a good occasion to make debug/asan not slow down and not block head builds (relevant whenever there is a new runner).

I'd be happy to help on this even if we merged the current PR. But I understand if you're not a big fan of making it ~slightly worse before we make it better :)

[eregon]

What if we introduced reuse of builds?

Yes that would be very interesting, I thought about it it recently but didn't try to implement it, and also something I might want to use for truffleruby-dev-builder for example.
Could you give it a shot?

[ivoanjo]

PR to add build reuse: #14

[ivoanjo]

Closing as superceded by #15