Skip to content

Merge mir query analysis invocations #140856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

oli-obk
Copy link
Contributor

@oli-obk oli-obk commented May 9, 2025

r? @ghost

same thing as #140854 just a different set of queries

Doing this in general has some bad cache coherence issues because the query caches are laid out in Vec lists per query where each index refers to a DefId in the same order as we're iterating. Iterating two or more lists at the same time does have cache issues, so I want to poke a bit at it to see if we can't merge just a few of them at a time.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 9, 2025
@oli-obk
Copy link
Contributor Author

oli-obk commented May 9, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 9, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request May 9, 2025
Merge mir query analysis invocations

r? `@ghost`

same thing as rust-lang#140854 just a different set of queries
@bors
Copy link
Collaborator

bors commented May 9, 2025

⌛ Trying commit d63b394 with merge bddede8...

@bors
Copy link
Collaborator

bors commented May 9, 2025

☀️ Try build successful - checks-actions
Build commit: bddede8 (bddede841a3c4fe1b7794ae75ad7036a512e5d30)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (bddede8): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.6% [0.6%, 0.6%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results (primary -0.9%, secondary -2.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.7% [0.4%, 1.1%] 3
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-1.1% [-2.7%, -0.5%] 20
Improvements ✅
(secondary)
-2.2% [-4.0%, -0.4%] 2
All ❌✅ (primary) -0.9% [-2.7%, 1.1%] 23

Cycles

Results (primary 1.5%, secondary 0.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.8% [0.4%, 3.9%] 26
Regressions ❌
(secondary)
4.5% [3.2%, 5.4%] 10
Improvements ✅
(primary)
-0.8% [-1.3%, -0.5%] 3
Improvements ✅
(secondary)
-7.0% [-7.8%, -6.3%] 6
All ❌✅ (primary) 1.5% [-1.3%, 3.9%] 29

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 773.559s -> 774.531s (0.13%)
Artifact size: 365.71 MiB -> 365.58 MiB (-0.03%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 9, 2025
@oli-obk
Copy link
Contributor Author

oli-obk commented May 9, 2025

@Zoxc do you think changes like this are detrimental to parallel rustc?

@Zoxc
Copy link
Contributor

Zoxc commented May 9, 2025

Merging parallel sections is generally good, but actual measurements are also nice.

@oli-obk oli-obk marked this pull request as ready for review May 10, 2025 07:39
@oli-obk
Copy link
Contributor Author

oli-obk commented May 10, 2025

Right. I'll figure out how to do that locally. But until then

r? @nnethercote

@oli-obk
Copy link
Contributor Author

oli-obk commented May 10, 2025

Ah no I forgot the regression only shows in cycles, not in instructions...

@oli-obk
Copy link
Contributor Author

oli-obk commented May 10, 2025

Branch misses and cache misses are also really badly regressed. So my original analysis still stands, even if wall time itself is not really affected

@@ -233,6 +222,17 @@ pub fn check_crate(tcx: TyCtxt<'_>) {
}
});

if tcx.features().rustc_attrs() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why dump attrs stuff later? I'm not opposed, just there's no motivation given.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to make space there to merge the parallel blocks before and after

@nnethercote
Copy link
Contributor

Branch misses and cache misses are also really badly regressed. So my original analysis still stands, even if wall time itself is not really affected

Can you remind me what the original analysis is? I've lost track of the motivation for this PR. Is it for code simplification, performance, both/something else? Thanks.

@oli-obk
Copy link
Contributor Author

oli-obk commented May 14, 2025

Doing this in general has some bad cache coherence issues because the query caches are laid out in Vec lists per query where each index refers to a DefId in the same order as we're iterating. Iterating two or more lists at the same time does have cache issues, so I want to poke a bit at it to see if we can't merge just a few of them at a time.

@oli-obk
Copy link
Contributor Author

oli-obk commented May 14, 2025

The reasons I'm doing it is that

  • merging those blocks allows for more parallelism as you don't run parallel blocks in sequence
  • merging blocks allows merging analysis queries shrinking the dep graph
  • should allow us to do more early aborting in case of errors and/or moving query calls from the analysis query into others that allow early aborting the others (and doing more tainting and stuff)

@nnethercote
Copy link
Contributor

The reasons I'm doing it is that

Aha, that helps a lot. Please put that explanation in the commit message for the third commit, and then r=me, thanks.

The reasons I'm doing it is that

* merging those blocks allows for more parallelism as you don't run parallel blocks in sequence
* merging blocks allows merging analysis queries shrinking the dep graph
* should allow us to do more early aborting in case of errors and/or moving query calls from the analysis query into others that allow early aborting the others (and doing more tainting and stuff)
@oli-obk
Copy link
Contributor Author

oli-obk commented May 14, 2025

@bors r=nnethercote

@bors
Copy link
Collaborator

bors commented May 14, 2025

📌 Commit e011c43 has been approved by nnethercote

It is now in the queue for this repository.

@bors bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 14, 2025
@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label May 14, 2025
@bors
Copy link
Collaborator

bors commented May 14, 2025

⌛ Testing commit e011c43 with merge 9e54713...

bors added a commit to rust-lang-ci/rust that referenced this pull request May 14, 2025
Merge mir query analysis invocations

r? `@ghost`

same thing as rust-lang#140854 just a different set of queries

Doing this in general has some bad cache coherence issues because the query caches are laid out in Vec<QueryResult> lists per query where each index refers to a DefId in the same order as we're iterating. Iterating two or more lists at the same time does have cache issues, so I want to poke a bit at it to see if we can't merge just a few of them at a time.
@rust-log-analyzer
Copy link
Collaborator

The job x86_64-msvc-1 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
test [rustdoc] tests\rustdoc\private\private-type-cycle-110629.rs ... FAILED
test [rustdoc] tests\rustdoc\private\private-use-decl-macro-47038.rs ... FAILED
test [rustdoc] tests\rustdoc\private\private-use.rs ... FAILED
test [rustdoc] tests\rustdoc\private\public-impl-mention-private-generic-46380-2.rs ... FAILED
##[error]There is not enough space on the disk. : 'C:\actions-runner\cached\_diag\blocks\cdae25ff-5147-4915-8b24-0e5cfc0e88cb_8e2ee900-9d77-5808-eab0-a63ec056909c.2'
##[error]There is not enough space on the disk. : 'C:\actions-runner\cached\_diag\pages\cdae25ff-5147-4915-8b24-0e5cfc0e88cb_7759ee95-cf8a-432e-8605-778761ce9794_1.log'

@bors
Copy link
Collaborator

bors commented May 14, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 14, 2025
@oli-obk
Copy link
Contributor Author

oli-obk commented May 14, 2025

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2025
@oli-obk
Copy link
Contributor Author

oli-obk commented May 14, 2025

@bors rollup=never

bors added a commit to rust-lang-ci/rust that referenced this pull request May 14, 2025
Merge mir query analysis invocations

r? `@ghost`

same thing as rust-lang#140854 just a different set of queries

Doing this in general has some bad cache coherence issues because the query caches are laid out in Vec<QueryResult> lists per query where each index refers to a DefId in the same order as we're iterating. Iterating two or more lists at the same time does have cache issues, so I want to poke a bit at it to see if we can't merge just a few of them at a time.
@bors
Copy link
Collaborator

bors commented May 14, 2025

⌛ Testing commit e011c43 with merge d8ccc90...

@bors
Copy link
Collaborator

bors commented May 14, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 14, 2025
@clubby789
Copy link
Contributor

System.IO.IOException: There is not enough space on the disk - @bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 14, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request May 14, 2025
Merge mir query analysis invocations

r? `@ghost`

same thing as rust-lang#140854 just a different set of queries

Doing this in general has some bad cache coherence issues because the query caches are laid out in Vec<QueryResult> lists per query where each index refers to a DefId in the same order as we're iterating. Iterating two or more lists at the same time does have cache issues, so I want to poke a bit at it to see if we can't merge just a few of them at a time.
@bors
Copy link
Collaborator

bors commented May 14, 2025

⌛ Testing commit e011c43 with merge ae6350f...

@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@bors
Copy link
Collaborator

bors commented May 14, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants