On Windows, "./miri many-seeds ./miri run" re-builds Miri over and over #3509
Description
Our many-seeds tests are extremely slow on Windows, taking something like 15s per iteration when it's <1s on other platforms. I think I figured out the underlying cause:
Trying seed: 1
$ cargo +miri build --bins --tests --locked --all-features --manifest-path D:\a\miri\miri\Cargo.toml
Compiling miri v0.1.0 (D:\a\miri\miri)
Finished `dev` profile [optimized + debuginfo] target(s) in 6.23s
$ cargo +miri build --bins --tests --locked --all-features --manifest-path D:\a\miri\miri\cargo-miri\Cargo.toml
Finished `dev` profile [optimized + debuginfo] target(s) in 0.06s
$ (building Miri sysroot)
$ cargo +miri run --locked --all-features --manifest-path D:\a\miri\miri\Cargo.toml -- -Zlayout-seed=1 -Zmiri-seed=1 tests/many-seeds/scoped-thread-leak.rs --edition=2021 --sysroot C:\Users\runneradmin\AppData\Local\rust-lang\miri\cache
Compiling miri v0.1.0 (D:\a\miri\miri)
Finished `dev` profile [optimized + debuginfo] target(s) in 6.31s
Running `target\debug\miri.exe -Zlayout-seed=1 -Zmiri-seed=1 tests/many-seeds/scoped-thread-leak.rs --edition=2021 --sysroot 'C:\Users\runneradmin\AppData\Local\rust-lang\miri\cache'`
There are two "Compiling Miri" here, for a single test invocation. This doesn't happen on other targets, e.g. on macOS:
Trying seed: 1
$ cargo +miri build --bins --tests --locked --all-features --manifest-path /Users/runner/work/miri/miri/Cargo.toml
Finished `dev` profile [optimized + debuginfo] target(s) in 0.22s
$ cargo +miri build --bins --tests --locked --all-features --manifest-path /Users/runner/work/miri/miri/cargo-miri/Cargo.toml
Finished `dev` profile [optimized + debuginfo] target(s) in 0.02s
$ (building Miri sysroot)
$ cargo +miri run --locked --all-features --manifest-path /Users/runner/work/miri/miri/Cargo.toml -- -Zlayout-seed=1 -Zmiri-seed=1 tests/many-seeds/scoped-thread-leak.rs --edition=2021 --sysroot /Users/runner/Library/Caches/org.rust-lang.miri
Finished `dev` profile [optimized + debuginfo] target(s) in 0.03s
Running `target/debug/miri -Zlayout-seed=1 -Zmiri-seed=1 tests/many-seeds/scoped-thread-leak.rs --edition=2021 --sysroot /Users/runner/Library/Caches/org.rust-lang.miri`
On Linux and macOS, while builds with and without "--tests" produce separate binaries (as some feature flags in dependencies change), it seems they get cached independently and once both are built, we can run both immediately. But on Windows, these two builds seem to clobber each other's cache.
Of course it's also silly to rebuild the sysroot each time, and if we avoided that we'd avoid switching between two different versions of the binary all the time. We'd still do some switching back and forth though, so it would still be better to have proper caching.
@epage is it expected that on Windows, builds of a binary crate with and without "--tests" re-build the binary each time, while on Unix they get cached properly? Or is there something else going on here? If I read the ls output on my machine correctly, then on Linux there are a bunch of miri-<hash> binaries and target/debug/miri is just a hardlink to the latest one. Windows doesn't have hardlinks so that exact same strategy will not work. And maybe there's a reason that just doing a copy instead of a rebuild does not work either?