master: sessions support broke --enable-heterogeneous builds

[jsquyres]

It looks like #9097 (6bd36a7) introduced a compile error when using --enable-heterogeneous. Both coll/adapt and pml/ob1 get compile errors. They both appear to stem from ob1/pml_ob1_hdr.h, so I'll only list a token set of compile errors from pml/ob1 here:

  CC       pml_ob1.lo
In file included from pml_ob1.h:42,
                 from pml_ob1.c:55:
pml_ob1_hdr.h: In function ‘ob1_hdr_ntoh’:
pml_ob1_hdr.h:532:6: error: implicit declaration of function ‘MCA_PML_OB1_EXT_MATCH_HDR_NTOH’; did you mean ‘MCA_PML_OB1_MATCH_HDR_NTOH’? [-Werror=implicit-function-declaration]
      MCA_PML_OB1_EXT_MATCH_HDR_NTOH(hdr->hdr_cid);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      MCA_PML_OB1_MATCH_HDR_NTOH
pml_ob1_hdr.h:534:30: error: ‘hext_hdr’ undeclared (first use in this function); did you mean ‘next_hdr’?
      ob1_hdr_ntoh (next_hdr, hext_hdr->hdr_common.hdr_type);
                              ^~~~~~~~
                              next_hdr
pml_ob1_hdr.h:534:30: note: each undeclared identifier is reported only once for each function it appears in
pml_ob1_hdr.h: In function ‘ob1_hdr_hton_intr’:
pml_ob1_hdr.h:587:6: error: implicit declaration of function ‘MCA_PML_OB1_EXT_MATCH_HDR_HTON’; did you mean ‘MCA_PML_OB1_MATCH_HDR_HTON’? [-Werror=implicit-function-declaration]
      MCA_PML_OB1_EXT_MATCH_HDR_HTON(hdr->hdr_cid);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      MCA_PML_OB1_MATCH_HDR_HTON
pml_ob1_hdr.h:589:30: error: ‘hext_hdr’ undeclared (first use in this function); did you mean ‘next_hdr’?
      ob1_hdr_hton (next_hdr, hext_hdr->hdr_common.hdr_type, proc);
                              ^~~~~~~~
pml_ob1_hdr.h:548:46: note: in definition of macro ‘ob1_hdr_hton’
     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
                                              ^
pml_ob1.c: In function ‘mca_pml_ob1_send_cid’:
pml_ob1.c:792:50: error: invalid type argument of ‘->’ (have ‘mca_pml_ob1_cid_hdr_t’ {aka ‘struct mca_pml_ob1_cid_hdr_t’})
     ob1_hdr_hton ((mca_pml_ob1_hdr_t *) &cid, cid->hdr_common.hdr_type, proc);
                                                  ^~
pml_ob1_hdr.h:548:46: note: in definition of macro ‘ob1_hdr_hton’
     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
                                              ^
cc1: some warnings being treated as errors
make: *** [pml_ob1.lo] Error 1

This obviously needs to be fixed before sessions can be brought to the v5.0.x branch.

[jsquyres]

#9982 did not solve the problem:

[6:44] savbu-usnic-a:~/g/o/o/m/p/ob1 (master)
$ make
  CC       pml_ob1.lo
In file included from pml_ob1.h:42,
                 from pml_ob1.c:55:
pml_ob1_hdr.h: In function ‘mca_pml_ob1_send_cid’:
pml_ob1_hdr.h:553:1: error: inlining failed in call to ‘always_inline’ ‘ob1_hdr_hton_intr’: function not considered for inlining
  553 | ob1_hdr_hton_intr(mca_pml_ob1_hdr_t *hdr, const uint8_t hdr_type,
      | ^~~~~~~~~~~~~~~~~
pml_ob1_hdr.h:551:5: note: called from here
  551 |     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pml_ob1_hdr.h:592:6: note: in expansion of macro ‘ob1_hdr_hton’
  592 |      ob1_hdr_hton (next_hdr, next_hdr->hdr_common.hdr_type, proc);
      |      ^~~~~~~~~~~~
make: *** [pml_ob1.lo] Error 1

[jsquyres]

@hppritcha FWIW, I did a configure with just --enable-heteorgeneous -- none of the debug configure flags, and it still fails with my gcc 10.2.0. Here's the compile line (here's my full config.log)

$ gcc --version
gcc (GCC) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ head config.log
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by Open MPI configure gitclone, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure --prefix=/home/jsquyres/bogus --enable-heterogeneous

$ cd ompi/mca/pml/ob1
$ make V=1
...
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../../../opal/include -I../../../../ompi/include -I../../../../oshmem/include -I../../../../ompi/mpiext/cuda/c -iquote../../../.. -I/home/jsquyres/git/ompi/3rd-party/libevent-2.1.12-stable -I/home/jsquyres/git/ompi/3rd-party/libevent-2.1.12-stable/include -I/home/jsquyres/git/ompi/3rd-party/hwloc-2.7.0/include -I/home/jsquyres/git/ompi/3rd-party/hwloc-2.7.0/include -I/home/jsquyres/git/ompi/3rd-party/openpmix/include -I/home/jsquyres/git/ompi/3rd-party/openpmix/include -O3 -DNDEBUG -Wundef -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wstrict-prototypes -Wcomment -Wshadow -Werror-implicit-function-declaration -fno-strict-aliasing -pedantic -Wall -Wformat-truncation=0 -finline-functions -mcx16 -MT pml_ob1.lo -MD -MP -MF .deps/pml_ob1.Tpo -c pml_ob1.c  -fPIC -DPIC -o .libs/pml_ob1.o
In file included from pml_ob1.h:42,
                 from pml_ob1.c:55:
pml_ob1_hdr.h: In function ‘ob1_hdr_hton_intr’:
pml_ob1_hdr.h:553:1: error: inlining failed in call to ‘always_inline’ ‘ob1_hdr_hton_intr’: recursive inlining
  553 | ob1_hdr_hton_intr(mca_pml_ob1_hdr_t *hdr, const uint8_t hdr_type,
      | ^~~~~~~~~~~~~~~~~
pml_ob1_hdr.h:551:5: note: called from here
  551 |     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pml_ob1_hdr.h:592:6: note: in expansion of macro ‘ob1_hdr_hton’
  592 |      ob1_hdr_hton (next_hdr, next_hdr->hdr_common.hdr_type, proc);
      |      ^~~~~~~~~~~~
pml_ob1_hdr.h: In function ‘mca_pml_ob1_send_cid’:
pml_ob1_hdr.h:553:1: error: inlining failed in call to ‘always_inline’ ‘ob1_hdr_hton_intr’: recursive inlining
  553 | ob1_hdr_hton_intr(mca_pml_ob1_hdr_t *hdr, const uint8_t hdr_type,
      | ^~~~~~~~~~~~~~~~~
pml_ob1_hdr.h:551:5: note: called from here
  551 |     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pml_ob1_hdr.h:592:6: note: in expansion of macro ‘ob1_hdr_hton’
  592 |      ob1_hdr_hton (next_hdr, next_hdr->hdr_common.hdr_type, proc);
      |      ^~~~~~~~~~~~
make: *** [pml_ob1.lo] Error 1

[jsquyres]

I should clarify that my above comment apparently automatically enabled debug options (due to the presence of the .git subdir). I note that when I do a VPATH build and all the debug stuff isn't enabled, I still get the compile error:

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../../../../ompi/mca/pml/ob1 -I../../../../opal/include -I../../../../ompi/include -I../../../../oshmem/include -I../../../../ompi/mpiext/cuda/c -iquote../../../../.. -iquote../../../.. -iquote../../../../../opal/include -iquote../../../../../ompi/include -iquote../../../../../oshmem/include -I/home/jsquyres/git/o3/build/3rd-party/libevent-2.1.12-stable -I/home/jsquyres/git/o3/build/3rd-party/libevent-2.1.12-stable/include -I/home/jsquyres/git/o3/build/3rd-party/hwloc-2.7.0/include -I/home/jsquyres/git/o3/3rd-party/hwloc-2.7.0/include -I/home/jsquyres/git/o3/build/3rd-party/openpmix/include -I/home/jsquyres/git/o3/3rd-party/openpmix/include -O3 -DNDEBUG -finline-functions -mcx16 -MT pml_ob1.lo -MD -MP -MF .deps/pml_ob1.Tpo -c ../../../../../ompi/mca/pml/ob1/pml_ob1.c  -fPIC -DPIC -o .libs/pml_ob1.o
In file included from ../../../../../ompi/mca/pml/ob1/pml_ob1.h:42,
                 from ../../../../../ompi/mca/pml/ob1/pml_ob1.c:55:
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h: In function ‘ob1_hdr_hton_intr’:
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h:553:1: error: inlining failed in call to always_inline ‘ob1_hdr_hton_intr’: recursive inlining
 ob1_hdr_hton_intr(mca_pml_ob1_hdr_t *hdr, const uint8_t hdr_type,
 ^~~~~~~~~~~~~~~~~
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h:551:5: note: called from here
     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h:592:6: note: in expansion of macro ‘ob1_hdr_hton’
      ob1_hdr_hton (next_hdr, next_hdr->hdr_common.hdr_type, proc);
      ^~~~~~~~~~~~
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h: In function ‘mca_pml_ob1_send_cid’:
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h:553:1: error: inlining failed in call to always_inline ‘ob1_hdr_hton_intr’: recursive inlining
 ob1_hdr_hton_intr(mca_pml_ob1_hdr_t *hdr, const uint8_t hdr_type,
 ^~~~~~~~~~~~~~~~~
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h:551:5: note: called from here
     ob1_hdr_hton_intr((mca_pml_ob1_hdr_t*)h, t, p)
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../../ompi/mca/pml/ob1/pml_ob1_hdr.h:592:6: note: in expansion of macro ‘ob1_hdr_hton’
      ob1_hdr_hton (next_hdr, next_hdr->hdr_common.hdr_type, proc);
      ^~~~~~~~~~~~
make: *** [pml_ob1.lo] Error 1

[bosilca]

The corresponding PR (#9993) has been merged.