Clippy and Miri should not block beta releases

[alexcrichton]

I think this is the fourth beta I'm having to re-figure-out how all this works and figure out how to ignore their failures on beta. @kennytm can we just exclude these tools from toolstate until this is figured out?

[oli-obk]

I'm slightly confused, why would these tools need to be disabled/their submodule update if they were building on master? Isn't the beta just the current master branch + some tweaks for --version output and no unstable features?

The release in March actually prevented the merge of any PRs on master that would break a tool. So I thought the idea was to fix all tools in that week.

Or is this about beta backports breaking tools?

[alexcrichton]

I don't think we've had a beta yet whether either clippy or miri were passing tests and checktool.sh on master fails the build if any tool fails to build on beta. We also do not want to release either of these tools to stable/beta yet.

[oli-obk]

We also do not want to release either of these tools to stable/beta yet.

neither of these tools has a Dist step, so there's no danger in that

I don't think we've had a beta yet whether either clippy or miri were passing tests

https://github.com/rust-lang/rust/pull/49589/files (the last beta) didn't have clippy issues. Since we have grace period of a week, I can easily get in a working clippy within that week (which then won't be broken until the beta). My issue is mainly that I don't really know when it begins ;)

Maybe we can ping the tool authors once a day that their tool needs to be fixed for the upcoming beta?

[alexcrichton]

My point with this issue is that we shouldn't automate ourselves into a hole where tools we do not want to ship block releases. That should not require authors to fix tools, we should simply ignore failures or have an easy way of disabling them. Right now producing a beta is pain as you have to resend it to bors multiple times after re-learning that these are blocking the release.

We do not ship miri/clippy but a successful clippy compilation affects how rls is compiled, which should not be happening on beta/stable.

[oli-obk]

We do not ship miri/clippy but a successful clippy compilation affects how rls is compiled, which should not be happening on beta/stable.

The code that influences rls is explicitly skipped on beta/stable, so the rls is completely free of clippy on beta/stable

That should not require authors to fix tools, we should simply ignore failures or have an easy way of disabling them.

for miri I agree, but for clippy, aren't we trying to move to a model where we actually dist it on stable, so I'd call having it ready for stable some form of proof of concept for the distributing part.

[alexcrichton]

Ok sure yes rls is protected but I feel like this is missing my point. Miri isn't ready. Clippy isn't ready. They're blocking beta releases. That shouldn't happen.