Improve MPI_Waitall performance for MPI_THREAD_MULTIPLE #9291
Conversation
Avoid atomic cmpxchng operations for MPI requests that are already complete. This improves the performance in message rate benchmarks. Signed-off-by: Austen Lauria <[email protected]>
|
I would like to see some benchmark data before this gets merged. Unfortunately, our master is currently broken for shared memory (which is the BTL that should show the most benefit with this patch). |
|
I was able to run NetPIPE-5.1 over shared memory on M1 and x86_64 and aside a certain level of variability the impact seems to be irrelevant (at best I am seeing about 1 nanosec difference). This is what I expected on a non contended memory location such as the request status. We need to keep in mind that contention on that specific memory location can only happen once, because at best the status is updated once by the thread that did the progress. |
|
After curating the data and looking more carefully at the diff between the 2 runs, there is indeed a trend toward a small positive impact in favor of this PR. The impact is 1 nano over about 170 on an M1, and around 1 nano over 300 on my Intel E5-2650 @ 2.3Ghz. |
|
Thanks @bosilca |
Avoid atomic cmpxchng operations for MPI requests that are already
complete. This improves the performance in message rate benchmarks.
Signed-off-by: Austen Lauria [email protected]