-
Notifications
You must be signed in to change notification settings - Fork 153
Run perf on Windows #834
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I don't personally have familiarity with Windows, but a key piece we'd need is to determine what tooling to use in place of |
I've started working on this. Based on these blog posts, it looks like we can capture a variety of hardware performance counters on Windows. I've also confirmed that this technique works and I can see instructions retired for rustc. Right now, I'm trying to get the collector building and running on Windows. I'm a bit curious @Mark-Simulacrum if you have any thoughts about how to organize the platform dependent code. |
Hm, so I think there are two big, related, questions here:
I'm going to operate on the assumption that the answer to the first is potentially yes, but the answer to the second is no - we don't actually want to be constantly comparing instruction counts on linux vs windows, for example. That means that likely no or very little work needs to be done on the frontend and database schema, and we can limit to the perf-stat invocation supporting Windows equivalents. I think my recommendation is that we don't try to add another perf collector, but rather substitute the backend in https://github.com/rust-lang/rustc-perf/blob/master/collector/src/rustc-fake.rs#L44 and try to mangle the relevant outputs in that file such that the majority of rustc-perf code doesn't even need to know it's running with "windows perf" vs. "linux perf". Does that sound plausible? Of course, if there are more bits that are linux-specific I'm not thinking of right now, we should talk about those. |
I'd like to see this land before we get too deep into discussing strategies of when to run the Windows tests. I agree that it's likely overkill to run them as often as we run Linux tests, but I do however want us to run them in some sort of automated way, and I do think it's interesting to run benchmarks that only make sense on Windows like winapi/windows-rs as often as we run the existing benchmarks. Of course, this all leads to interesting questions of which hardware we'll use to run these tests. |
I think that would be great but we'll need to explore how to make that actually happen of course.
I think there is some value in "doubling up" some of the crates, not to compare Linux vs Windows but just to make sure we aren't introducing regressions that affect Windows disproportionately for some reason. For now though, I would be very happy just to get windows-rs being benchmarked.
Yeah, I agree. I'm focusing on just being able to invoke the collector on Windows.
I was more referring to stuff like this: rustc-perf/collector/src/execute.rs Lines 26 to 68 in c52ee62
where we shell out to Linux utilities. If it's feasible, I'm planning to convert these to Rust code or otherwise use standard Windows tools. I would rather not require userland Linux tools to be available to run the collector. |
Yes, those should all likely be converted to Rust code - if something proves difficult to convert, we can continue shelling out to a platform-specific utility but hopefully that's not necessary for the most part. I agree that getting support landed for running the collector manually is the first priority, just wanted to communicate that if we wanted to extend the official collection - particularly to double up testing - we'd likely need to do some database/frontend work as well, depending on the strategy used. |
Uh oh!
There was an error while loading. Please reload this page.
It would be useful to also have the ability to run the perf suite on Windows. One open question would be whether this would run as often as the Linux tests for or if we would kick off Windows tests at a different cadence or on-demand.
This might also help with #181
There's a few different things we'll need to do to fully complete this issue:
--self-profile
) Add support for building rustc-perf and running the collector on Windows #865xperf + ETW
to capture hardware performance counter metrics like instructions retired. Support collecting hardware performance counters on Windows #885win-rs
or a subset of it might be a good candidate).Nice to haves:
cpu-clock
perf pseudo-counter #897The text was updated successfully, but these errors were encountered: