RMA Functionality #7813
Description
I'm using MPI 4.0.1 installed from a system module. Our cluster is running RHEL6 with an infiniband network. The cores are Intel Xeon processors.
I’m curious about the status of RMA functionality in OpenMPI.
We’ve had a MPI-RMA based runtime in Global Arrays for some time now but it does not work well with OpenMPI. It does, however, operate reasonably well with other MPI implementations. I’ve run the GA test suite using the MPI-RMA runtime against different implementations of MPI-3 and the OpenMPI implementation performs significantly worse than implementations from MPICH, Intel and MVAPICH. The current test suite contains 80 programs. Using Intel MPI/5.1.3.181 we get 3 failures, with MPICH 3.3.2 we get 8 failures (MPICH 3.4a2 reduces this to 3 failures) and MVAPICH 2.3.2 reports 9 failures. In comparison, OpenMPI 4.0.1 reports 47 failures. We do have a few tests that are problematic across all implementations and we are looking at them from the GA side.
Our standard test configuration is to run on 4 processors split between two nodes. This forces at least some of the communication on to the switch. I can provide additional details about configuring and running the GA test suite as well as what tests are failing, if you are interested.
Given that we get much better results with other implementations, we believe the large number of failures we are seeing with OpenMPI are reflective of problems in the OpenMPI implementation of RMA functions in MPI-2 and 3. Are there any plans to improve these implementations? We have alternative implementations based on traditional 2-sided MPI that are robust and reasonably high performing, but we are still interested in using runtimes that sit directly on top of MPI RMA constructs. Are there plans going forward to aggressively improve the RMA implementations?