Something with ./miri install or the miri toolchain is mangling itself? Or rustup?

[saethlin]

I've had issues with ./miri install for a long time. Perhaps this is just a me problem? I have very little interest in just running ./miri code.rs to execute Rust code because all the interesting cases for me are whole Cargo projects. So I need some way to run cargo-miri on Cargo projects to do development, and I've settled on ./miri install then cargo +miri miri.

After I rustup update to pull in the latest nightly, cargo +miri miri run doesn't work, because somehow miri becomes uninstalled. I need to run ./miri install again.

Recently, I started getting this behavior when I try to cargo +miri miri run:

    <miri compiling the standard library>
    Checking getopts v0.2.21
    Checking test v0.0.0 (/home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test)
    Finished release [optimized] target(s) in 1.02s
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/ben/.cargo/bin/cargo-miri /home/ben/.rustup/toolchains/miri/bin/rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit status: 1)
  --- stderr
  fatal error: file "/home/ben/.rustup/toolchains/miri/bin/rustc" contains outdated or invalid JSON; try `cargo clean````

Sometimes running rustup toolchain remove miri then ./miri install twice fixes this, but not anymore. I've tried the cargo clean as suggested as well as rm -r ~/.cache/miri, those do not seem to have any effect (other than requiring recompiling the standard library).

I also just ran into this situation after a rustup update:

╭ ➜ ben@archlinux:/tmp/scratch
╰ ➤ MIRIFLAGS=-Zmiri-preemption-rate=1 cargo +miri miri run
I will run `"rustup" "component" "add" "rust-src"` to install the `rust-src` component for the selected toolchain. Proceed? [Y/n] y
error: miri is a custom toolchain
fatal error: failed to install the `rust-src` component for the selected toolchain
╭ ➜ ben@archlinux:/tmp/scratch
╰ ➤ MIRIFLAGS=-Zmiri-preemption-rate=1 cargo +miri miri run
I will run `"rustup" "component" "add" "rust-src"` to install the `rust-src` component for the selected toolchain. Proceed? [Y/n] y
error: miri is a custom toolchain
fatal error: failed to install the `rust-src` component for the selected toolchain
╭ ➜ ben@archlinux:~
╰ ➤ cd ~/miri
╭ ➜ ben@archlinux:~/miri
╰ ➤ ./rustup-toolchain 
miri toolchain is already at commit 99930ac7f8cbb5d9b319b2e2e92794fd6f24f556.
info: override toolchain for '/home/ben/miri' set to 'miri'

So I rustup toolchain remove miri then run ./rustup-toolchain and I watch the toolchain install and get exactly the same behavior. So I figured I would try out nightly:

╭ ➜ ben@archlinux:/tmp/scratch
╰ ➤ cargo +nightly miri run
I will run `"rustup" "component" "add" "rust-src"` to install the `rust-src` component for the selected toolchain. Proceed? [Y/n] y
info: component 'rust-src' is up to date
fatal error: given Rust source directory `lib/rustlib/src/rust/library` does not exist.

None of this makes any sense to me.

So I just got myself out of this situation by running rustup self uninstall and doing a full reinstall from scratch. Things work again, for the moment. Is there a better way to do cargo-miri development?

[RalfJung]

The issue is that rustup and ./miri install are fighting for who controls the miri and cargo-miri binaries. Every time you run rustup update, it will basically destroy your hand-installed Miri. Some of your errors might arise from the two binaries being in a mixed state, where one stems from rustup and ones from your hand-installed Miri. That cannot go well.

What I do is I always run ./miri install just before I want to run my locally built miri via cargo +miri miri. That has worked 100% reliably for me.

At the very least, you will need to re-do ./miri install

• each time you run rustup update
• each time the miri toolchain changes

Doing cargo +nightly miri only makes sense if you are not using a hand-installed Miri, but want to use the rustup-supplied version. There probably is some way to tell rustup to reinstall all its binary wrappers (i.e., to definitely get rid of the hand-installed Miri), but I don't know it.

[saethlin]

I figured out that I needed to run ./miri install every time something changed with rustup, but that stopped working which is why I opened this.

A lot of the output above I find very confusing. Why does cargo-miri want to read JSON from the toolchain's rustc executable? That's an ELF file on my setup.

Why after running rustup toolchain remove miri then ./rustup-toolchain then ./miri install did I somehow end up with a cargo-miri which thought it needed to install rust-src?

[RalfJung]

Everything still works as expected on my system. Do you have a series of commands that you think should work, but doesn't, to reproduce the problem?

Why does cargo-miri want to read JSON from the toolchain's rustc executable? That's an ELF file on my setup.

This has something to do with how cargo-miri works. The "storing the flags" that it mentions means "put them into a JSON file". So somehow it thinks that it should "execute" that file, and "execute" for cargo-miri means "interpret as JSON and then run the Miri driver".

Why after running rustup toolchain remove miri then ./rustup-toolchain then ./miri install did I somehow end up with a cargo-miri which thought it needed to install rust-src?

My guess would be it is run in the wrong toolchain: cargo +nightly miri run. Then the paths end up being all messed up so it doesn't find things.

[saethlin]

I don't have a reproducer. I wish I did, but rustup has so much global state, so I'm mostly just hoping I manage to run into this issue again.

With cargo +miri miri run I was getting this error:

fatal error: file "/home/ben/.rustup/toolchains/miri/bin/rustc" contains outdated or invalid JSON; try `cargo clean`

As a last-ditch effort to get some more info before I reinstalled rustup, I tried with +nightly because some other people have told me that their cargo +nightly miri still works after a ./miri install.

[RalfJung]

I wonder if #2259 will help here? Some of the symptoms look the same as #2238 (comment).

[RalfJung]

@saethlin is there still anything actionable in this issue? It doesn't seem reproducible, we improved our docs for how to recover from ./miri install and give control back to rustup, cargo +nightly miri after ./miri install should now show a reasonable error message complaining about a toolchain mismatch, and #2259 fixed a problem with similar symptoms.

[saethlin]

As above, we left this open in the hope that it would recur. It has not. I'm not sure if that's good or bad 😂