Configure script wrongly assumes -fPIC is not required in fortran integer size test

[haampie]

This is an exact copy of an issue I had compiling MPICH.

When running configure on Open MPI v4.1.2 with clang as a C / C++ compiler and gcc as a Fortran compiler, I'm hitting the following:

configure:45467: clang -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -I. -c conftest.c
configure:45474: $? = 0
configure:45484: gfortran   conftestf.f90 conftest.o -o conftest   -L/path/to/zlib/lib  -lz
/usr/bin/ld: conftest.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE

The solution is to either drop -O3 from clang or add -fPIE to clang.

See here how MPICH solves it pmodels/mpich#5771 for reference.

---

See here for a reproducer:

conftest.c

#include <stdio.h>
int cisize_(char *,char*);
int cisize_(char *i1p, char *i2p) {
    int isize_val=0;
    FILE *f = fopen("conftestval", "w");
    if (!f) return 1;
    isize_val = (int)(i2p - i1p);
    fprintf(f,"%d\n", isize_val);
    fclose(f);
    return 0;
}

conftest.f

            program main
            integer a(2)
            integer irc, cisize
            irc = cisize(a(1),a(2))
            end

this fails

$ clang -O2 -c conftest.c && gfortran -o conftest conftest.f conftest.o
/usr/bin/ld: conftest.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

this works

$ clang -fPIC -c conftest.c && gfortran -o conftest conftest.f conftest.o
$ clang -c conftest.c && gfortran -o conftest conftest.f conftest.o

[jsquyres]

@haampie Can you cite what versions of the compilers and/or platforms you're using? I ask because I'm running into difficulty trying to replicate on x86-64 RHEL 7 with recent-ish clang / gfortran:

$ clang --version
clang version 10.0.1 
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /cm/shared/apps/clang/10.0.1/rhel7/bin
$ gfortran --version
GNU Fortran (GCC) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ clang -O2 -c conftest.c && gfortran -o conftest conftest.f conftest.o
$ clang -fPIC -c conftest.c && gfortran -o conftest conftest.f conftest.o
$ clang -c conftest.c && gfortran -o conftest conftest.f conftest.o
$ 

[haampie]

Quickly tried on a few versions of Ubuntu:

Works:

• Ubuntu 16.04 with clang 3.8.0 and gcc 5.4.0

Fails:

• Ubuntu 18.04 with clang 6.0.0 and gcc 7.5.0
• Ubuntu 20.04 with clang 10.0.0 and gcc 9.3.0
• Ubuntu 21.10 with clang 13.0.0 and gcc 11.2.0

Reproducer:

$ docker run --rm -v $PWD:$PWD -w $PWD ubuntu:20.04 sh -c 'apt-get update -qqy > /dev/null && apt-get install --no-install-recommends -qqy clang gfortran > /dev/null && clang -O2 -c conftest.c && gfortran -o conftest conftest.f conftest.o'
/usr/bin/ld: conftest.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
collect2: error: ld returned 1 exit status

[jsquyres]

I think the problem is a bit bigger than this, for a few reasons.

I'm still unable to replicate your results when I use the same versions as you of clang and gfortran on RHEL 7. These are clang and gcc/gfortran that I have built myself, and therefore don't have whatever OS package/build defaults you're getting on the Ubuntu images.

Of course, I can replicate your results on the Ubuntu/docker command you provided.

This says to me that there's something in the config of clang and/or gfortran at play here -- not just versions of the compilers. I do not know exactly what this is; it could be that the clangs I have built manually are defaulting to building PIC code...? (I've unfortunately spent a bunch of time on this already and don't have more time to chase down what the differences are)

Additionally, even if I update Open MPI's configure to use $CFLAGS_WITHOUT_OPTFLAGS for all the C/Fortran tests (which, as the variable name implies, drops the -O2 or -O3 with the C compilation and therefore things work out, even in the Docker container you provided), we still run into problems with Open MPI's thread tests in configure where we're trying to figure out what thread flags are needed for the Fortran compiler. This does a similar thing as the previous tests: compile and link C and Fortran code together. But in this example, the C code is compiled without optimization flags, yet we still get the relocation R_X86_64_32 ... linker error. That is confusing to me, and I've unfortunately run out of time for the moment in tracking down how this C+Fortran configure test is different than the previous C+Fortran configure tests (where simply removing the optimization flag is sufficient).

Adding -fpic to the C compilation part of the Fortran thread test makes this work, but it makes me suspicious of the whole premise here.

Finally, even if I manually hard-code a -fpic into the Fortran thread test in configure and successfully complete configure, build Open MPI, and install Open MPI, then if I do a similar test:

• Use mpicc to compile c_code.c to c_code.o
• Use mpifort to compile fortran_code.f and link it to c_code.o

I'll get the same relocation ... error. I didn't look too closely at the MPICH solution, but I suspect the same may be true there, too...? That would be worth checking, and notifying the MPICH devs if there's still a problem there.

Regardless, I think the solution you want may be as simple as:

$ ./configure CFLAGS=-fpic CC=clang FC=gfortran --with-wrapper-cflags=-fpic --with-wrapper-cxxflags=-fpic ....`

[bwbarrett]

On Ubuntu 20.04 (I didn't try the docker container above, but bare hardware), it would appear that the GNU compiler suite packaged by Ubuntu defaults to generating PIC (and therefore PIE) code, while the LLVM compiler suite packaged by Ubuntu does not. My test program is:

#ifdef __pic__
#warning "pic"
#endif

#ifdef __pie__
#warning "pie"
#endif

int func(void)
{
    return 5;
}

With GCC 9.3.0:

% gcc foo.c -c -o foo.o
foo.c:2:2: warning: #warning "pic" [-Wcpp]
    2 | #warning "pic"
      |  ^~~~~~~
foo.c:6:2: warning: #warning "pie" [-Wcpp]
    6 | #warning "pie"
      |  ^~~~~~~
% gcc -fno-pic foo.c -c -o foo.o
%

With LLVM 10.0.0:

% clang-10 foo.c -c -o foo.o
% clang-10 -fpic foo.c -c -o foo.o
foo.c:2:2: warning: "pic" [-W#warnings]
#warning "pic"
 ^
1 warning generated.
% clang-10 -fpie foo.c -c -o foo.o
foo.c:2:2: warning: "pic" [-W#warnings]
#warning "pic"
 ^
foo.c:6:2: warning: "pie" [-W#warnings]
#warning "pie"
 ^
2 warnings generated.
%

So this is really a systems problem; Ubuntu shouldn't have the two compiler suites have different defaults for generating PIC/PIE code. It means that any time you're mixing object files (rather than using libraries) between the two compiler suites, you're going to run into trouble. Obviously, this problem is most evident in your case, because flang isn't where you want to be these days.

Jeff's answer is right; this isn't a problem that Open MPI should be solving (blindly setting -fPIC has its own problems, and since this has come up once in 20 years, it is clearly not the common case). Given the easy workaround of setting CFLAGS/CXXFLAGS given the known situation you're in, that's the right answer.

[haampie]

Thanks for double checking, I'll mention this thread with the MPICH folks too.

Turns out, there's a report about this from 2016: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847554 but I don't see any follow up :(