Recompress xz -> xz with tuned compression options #58
Conversation
Mark-Simulacrum
force-pushed
the
recompress-xz
branch
from
01c9a66 to
f56c4cb
Compare
|
For additional context I decided to work on this since I'm seeing some of our fastest builders take ~1/2 their time on the single-threaded recompression (e.g., see CPU usage for dist-android). This isn't a direct problem for our overall CI times, but it does mean that they're using relatively expensive builders -- the 9 minutes of compression cost us $0.576/build. Across tens of short builders that adds up quickly. CodeBuild is cheaper per-vCPU ($0.0025/vCPU/minute vs. $0.004/vCPU/minute on GHA) and since we're using all available CPUs for recompression we're not wasting 15/16 cores we're paying for as well. We also pay the cost once/day rather than 7 times per day. All-around this just makes more sense. We do pay a 5-15% more for CI artifact storage costs, but that's not a huge concern at this time, storage is cheap.
|
Mark-Simulacrum
force-pushed
the
recompress-xz
branch
2 times, most recently
from
7797c6e to
6e92fbb
Compare
This is a companion PR to rust-lang/promote-release#58, which moves the relevant optimal code to rust-lang/promote-release. As mentioned in the comments of that PR, this is expected to cut CI costs (and time, though predominantly felt on fast builders) and reduce wasted resources due to in-practice single-threaded compression not using the full 8+ vCPU builders we have available.
This adds support for recompressing the .xz tarballs in promote-release, with a goal of moving rust-lang/rust CI back to producing "balanced" profile binaries rather than the highly compressed ones currently produced. This will cut rust-lang/rust CI times while keeping the production artifacts in static.r-l.o equivalently compressed. This PR doesn't enable the .xz recompression by default, that will be done in a follow up simpleinfra push. Depending on the impact to (especially) stable build times we may wish to bump the promote-release production container from 8 cores to 72 cores (the next largest container) so that we still finish in a reasonable amount of time.
Mark-Simulacrum
force-pushed
the
recompress-xz
branch
from
6e92fbb to
55983f3
Compare
This is pending upstream PR rust-lang/promote-release#58 but is harmless to land ahead of time.
|
Would it make sense to go further and have CI only build zstd tarballs and do a zstd -> xz conversion only for static.r-l.o? |
|
Perhaps, but if we stop producing XZ tarballs in CI that'll need a bunch of tools to change (cargo-bisect-rustc, crater, perf, etc.) and so needs additional work in a bunch of places. I suspect that's the right long-term trajectory but I expect this to already be a significant win. |
…, r=pietroalbini Remove optimal xz settings from CI This is a companion PR to rust-lang/promote-release#58, which moves the relevant optimal code to rust-lang/promote-release. As mentioned in the comments of that PR, this is expected to cut CI costs (and time, though predominantly felt on fast builders) and reduce wasted resources due to in-practice single-threaded compression not using the full 8+ vCPU builders we have available. This probably shouldn't land before that PR + a simpleinfra change to enable the recompression of xz artifacts. But if it does land, it's just a matter of a few nightlies with slightly larger artifacts, so not a big deal. r? `@pietroalbini`
…oalbini Remove optimal xz settings from CI This is a companion PR to rust-lang/promote-release#58, which moves the relevant optimal code to rust-lang/promote-release. As mentioned in the comments of that PR, this is expected to cut CI costs (and time, though predominantly felt on fast builders) and reduce wasted resources due to in-practice single-threaded compression not using the full 8+ vCPU builders we have available. This probably shouldn't land before that PR + a simpleinfra change to enable the recompression of xz artifacts. But if it does land, it's just a matter of a few nightlies with slightly larger artifacts, so not a big deal. r? `@pietroalbini`
This adds support for recompressing the .xz tarballs in promote-release, with a goal of moving rust-lang/rust CI back to producing "balanced" profile binaries rather than the highly compressed ones currently produced. This will cut rust-lang/rust CI times while keeping the production artifacts in static.r-l.o equivalently compressed. The settings in this PR match the current configuration in rust-lang/rust's best compression profile.
This PR doesn't enable the .xz recompression by default, but that will happen in production as I've deployed the env variable with =1 already (we can remove that if we run into trouble in production easily). Depending on the impact to (especially) stable build times we may wish to bump the promote-release production container from 8 cores to 72 cores (the next largest container) so that we still finish in a reasonable amount of time. This is done so that we can land this PR immediately and then follow-up with necessary changes in rust-lang/rust and in simpleinfra.
r? @pietroalbini