Missing ELEMENTAL in declaration of ompi_request_op_ne in mpi_f08 module?

[sfilippone]

Thank you for taking the time to submit an issue!

Background information

What version of Open MPI are you using? (e.g., v3.0.5, v4.0.2, git branch name and hash, etc.)

openmpi-4.1.0-5.fc34.x86_64

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Fedora 34 package

If you are building/installing from a git clone, please copy-n-paste the output from git submodule status.

Please describe the system on which you are running

• Operating system/version:
• Computer hardware:
• Network type:
  Linux laptop

---

Details of the problem

When using the MPI module mpi_f08 I am getting the following error.

psi_dswapdata.F90:248:16:

  248 |         if (any(y%comid .ne. mpi_request_null)) then
      |                1
Error: Operands of comparison operator ‘.ne.’ at (1) are TYPE(mpi_request)/TYPE(mpi_request)

The y%comid variable is an array.
In mpi-types.F90 (am not sure how the various includes work with the f08 version) I see

 logical function ompi_request_op_ne(a, b)
    type(MPI_Request), intent(in) :: a, b
    ompi_request_op_ne = (a%MPI_VAL .NE. b%MPI_VAL)
  end function ompi_request_op_ne

Now, the error message above shold be fixed by declaring the function to be ELEMENTAL so that the array comparison is treated properly (and similarly for all the other defined equality/inequality operators)

 elemental  function ompi_request_op_ne(a, b) result(res)
   logical :: res 
    type(MPI_Request), intent(in) :: a, b
    res = (a%MPI_VAL .NE. b%MPI_VAL)
  end function ompi_request_op_ne

[jsquyres]

Forgive my ignorance -- what does it mean to compare an array to a scalar?

[sfilippone]

The statement is ANY(y%comid /= mpi_request_null) A comparison between an array and a scalar gives on output an array of LOGICALS. The ANY is a reduction that gives on output the scalar TRUE if at least one of the comparisons is TRUE. Hence, the output of ANY() (or of ALL() for that matter, with the obvious meaning) can be used in an IF statement. Cheers S.

…

On Tue, Mar 1, 2022 at 3:51 PM Jeff Squyres ***@***.***> wrote: Forgive my ignorance -- what does it mean to compare an array to a scalar? — Reply to this email directly, view it on GitHub <#10057 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD274T7K5A26OFAILEC3IJTU5YVGDANCNFSM5PUCTS2A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jsquyres]

Interesting. Ok. I have some followup questions:

1. Does using ELEMENTAL violate the MPI spec? I don't think so -- it doesn't look like the MPI 4.0 spec says anything about ELEMENTAL at all. Indeed, MPI-4.0 section 19.1.2, "Fortran Support Through the mpi_f08 Module", only says the following about the operator functions (p793, 22:23):

Overload the operators .EQ. and .NE. to allow the comparison of these MPI handles with .EQ., .NE., == and /=.

2. What will adding ELEMENTAL to the mpi_f08 bindings in the middle of a release series (e.g., in 4.1.2) mean towards ABI guarantees for the v4.x series? E.g., will that change the back-end symbol created for the function by the Fortran compiler?

I note that MPICH doesn't seem to use ELEMENTAL for its handle operator functions, either. If we make this change, we should probably encourage MPICH to make the change, too (for application portability).

[jsquyres]

FWIW: I just asked this question to the MPI Forum Fortran working group (which hasn't needed to be active much lately, but they still respond to questions) to get some advice, too: https://lists.mpi-forum.org/pipermail/mpiwg-fortran/2022-March/003613.html.

[sfilippone]

ELEMENTAL means that the Fortran compiler is allowed to apply that function element-by-element (in the context of an array expression). That is what the compiler does with the "normal" .EQ. and .NE. operators; given that these functions are very simple and can be declared ELEMENTAL, I would go for the behaviour that generates the least surprise. I do not know whether ELEMENTAL changes the ABI; as far as I can tell, the symbol generated by GNU Fortran is the same. As for MPICH, I sent a bug report to the discuss mailing list with the same contents. Cheers S.

…

On Tue, Mar 1, 2022 at 4:22 PM Jeff Squyres ***@***.***> wrote: Interesting. Ok. I have some followup questions: 1. Does using ELEMENTAL violate the MPI spec? I don't *think* so -- it doesn't look like the MPI 4.0 spec says anything about ELEMENTAL at all. Indeed, MPI-4.0 section 19.1.2, "Fortran Support Through the mpi_f08 Module", only says the following about the operator functions (p793, 22:23): Overload the operators .EQ. and .NE. to allow the comparison of these MPI handles with .EQ., .NE., == and /=. 1. What will adding ELEMENTAL to the mpi_f08 bindings in the middle of a release series (e.g., in 4.1.2) mean towards ABI guarantees for the v4.x series? E.g., will that change the back-end symbol created for the function by the Fortran compiler? I note that MPICH doesn't seem to use ELEMENTAL for its handle operator functions, either. If we make this change, we should probably encourage MPICH to make the change, too (for application portability). — Reply to this email directly, view it on GitHub <#10057 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD274T3RR23RZBWTSTW7TNTU5YYZXANCNFSM5PUCTS2A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jsquyres]

Let's link up all 3 discussions (this Open MPI github issue, the MPICH discuss thread, and the Forum Fortran working group discussion) so that we don't have to have the same Q&A 3 times.

Where's the MPICH discuss thread? I don't see it in the archives: https://lists.mpich.org/pipermail/discuss/ and I haven't received it in my INBOX yet.

[jsquyres]

If your mail to discuss@mpich.org was set to moderation as a first-time poster, perhaps it would be better to cancel that message and file a github issue at https://github.com/pmodels/mpich/issues, and link to this issue and the Forum Fortran WG thread...?

[sfilippone]

If your mail to discuss@mpich.org was set to moderation as a first-time poster, perhaps it would be better to cancel that message and file a github issue at https://github.com/pmodels/mpich/issues, and link to this issue and the Forum Fortran WG thread...?

Ok, will do.

[sfilippone]

Interesting. Ok. I have some followup questions:

1. Does using `ELEMENTAL` violate the MPI spec?  I don't _think_ so -- it doesn't look like the MPI 4.0 spec says anything about `ELEMENTAL` at all.  Indeed, MPI-4.0 section 19.1.2, "Fortran Support Through the mpi_f08 Module", only says the following about the operator functions (p793, 22:23):

Overload the operators .EQ. and .NE. to allow the comparison of these MPI handles with .EQ., .NE., == and /=.

2. What will adding `ELEMENTAL` to the `mpi_f08` bindings in the middle of a release series (e.g., in 4.1.2) mean towards ABI guarantees for the v4.x series?  E.g., will that change the back-end symbol created for the function by the Fortran compiler?

I note that MPICH doesn't seem to use ELEMENTAL for its handle operator functions, either. If we make this change, we should probably encourage MPICH to make the change, too (for application portability).

Some additional background:

1. The issue surfaced when moving from MPI_MOD/integer handles to MPI_F08/type(mpi_request), and the code stopped working, whereas the expectation would be that moving from an intrinsic comparison to a defined operator should not cause an error;
2. The ELEMENTAL declaration carries with it some restriction, all of them are easily satisfied by the operators' code
3. Adding ELEMENTAL would allow some additional cases to be handled, but would not in any way break existing source code, it would still compile just fine;
4. As you correctly pointed out, the ABI might be a different matter; I think this is not going to be an issue, but I guess someone more familiar with compiler internals would need to confirm
5. Binary operators on arrays in Fortran are interpreted as element-by-element and require the arrays to have the same shape, except that scalars are always conformant to any arrays and are treated as if they were replicated as many times as necessary; compare this with the "dotted" operators in Matlab (the Matlab array language is almost the same as the Fortran array language)

[hzhou]

4. As you correctly pointed out, the ABI might be a different matter;

From my limited knowledge, Fortran doesn't have a well-defined ABI to start with. The module interface (where the keyword elemental is being discussed) is all compiler-specific and may well be just a gzip-ed source code.

[jsquyres]

@hzhou I was referring to Open MPI's ABI guarantees. I.e., if Open MPI v4.1.0 was built with fortran compiler X and compiler options Y and configure options Z, then an MPI application -- even an application that uses mpi_f08 -- built against that Open MPI installation should work without recompilation with the shared libraries of another Open MPI v4.1.x installation that was built the same way.

Hence, if ELEMENTAL changes something in terms of the back-end symbol(s) that the compiler generates, that will complicate things. I have not had the cycles to check this.