Build failure on macOS Big Sur

[carlocab]

Building from the latest commit on the v2.1 branch on macOS Big Sur produces the following error:

❯ make clean && make
/Library/Developer/CommandLineTools/usr/bin/make -C src clean
rm -f luajit libluajit.a libluajit.so host/minilua host/buildvm lj_vm.S lj_bcdef.h lj_ffdef.h lj_libdef.h lj_recdef.h lj_folddef.h host/buildvm_arch.h jit/vmdef.lua *.o host/*.o *.obj *.lib *.exp *.dll *.exe *.manifest *.pdb *.ilk
==== Building LuaJIT 2.1.0-beta3 ====
/Library/Developer/CommandLineTools/usr/bin/make -C src
HOSTCC    host/minilua.o
HOSTLINK  host/minilua
DYNASM    host/buildvm_arch.h
HOSTCC    host/buildvm.o
HOSTCC    host/buildvm_asm.o
HOSTCC    host/buildvm_peobj.o
HOSTCC    host/buildvm_lib.o
HOSTCC    host/buildvm_fold.o
HOSTLINK  host/buildvm
BUILDVM   lj_vm.S
ASM       lj_vm.o
CC        lj_assert.o
CC        lj_gc.o
BUILDVM   lj_ffdef.h
CC        lj_err.o
lj_err.c:469:2: error: "Broken build system -- only use the provided Makefiles!"
#error "Broken build system -- only use the provided Makefiles!"
 ^
1 error generated.
make[1]: *** [lj_err.o] Error 1
make: *** [default] Error 2

I have MACOSX_DEPLOYMENT_TARGET=11.1 set.

I attempted a git bisect, which lead to the following commit:

e131936133c58de4426c595db2341caf5a1665b5 is the first bad commit
commit e131936133c58de4426c595db2341caf5a1665b5
Author: Mike Pall <mike>
Date:   Tue Mar 23 00:22:34 2021 +0100

    Cleanup and enable external unwinding for more platforms.

 doc/extensions.html |  22 +---
 src/Makefile        |  11 +-
 src/lj_arch.h       |  33 +++---
 src/lj_err.c        | 321 +++++++++++++++++++++++++++-------------------------
 4 files changed, 197 insertions(+), 190 deletions(-)

However, the commit can't be cleanly reverted, so it's not quite clear to me whether that is, in fact, the source of the problem.

[MikePall]

It looks like the build process doesn't recognize that the OSX toolchain generates unwind tables.

The code in question is in src/Makefile and does approximately the following:

echo 'extern void b(void); int a(void) { b(); return 0; }' >tmpunwind.c
gcc -O2 -c tmpunwind.c -o tmpunwind.o
grep -qa -e eh_frame -e __unwind_info tmpunwind.o && echo HAS UNWIND INFO

So either the compiler doesn't add a __unwind_info section to the object file or OSX grep is unable to search in a binary file.

Could you please check what happens, if you do this by hand? E.g. whether tmpunwind.o really contains an __unwind_info secion and/or whether it's the grep that fails?

If the former doesn't work, then I'll need a different way to verify the toolchain actually produces unwind info. Maybe we can change the test C code slightly.

[carlocab]

It might be macOS grep:

❯ echo 'extern void b(void); int a(void) { b(); return 0; }' >tmpunwind.c
gcc -O2 -c tmpunwind.c -o tmpunwind.o
grep -qa -e eh_frame -e __unwind_info tmpunwind.o && echo HAS UNWIND INFO
❯ echo $?
1

However:

❯ otool -l tmpunwind.o | grep -e eh_frame -e __unwind_info
  sectname __eh_frame

(I hope I'm looking at the correct thing.)

[MikePall]

OK, so I hardcoded this for OSX. Fix committed. Thanks!

[carlocab]

Great; that worked. Thanks for the lightning-quick response.

[carlocab]

By the way, I looked into your grep test. I think the issue is that macOS uses BSD grep, which requires slightly different flags. In particular, the following test works:

❯ echo 'extern void b(void); int a(void) { b(); return 0; }' >tmpunwind.c
gcc -O2 -c tmpunwind.c -o tmpunwind.o
grep -qU -e eh_frame -e __unwind_info tmpunwind.o && echo HAS UNWIND INFO
HAS UNWIND INFO

It seems you need the -U flag to search binaries with BSD grep:

     -U, --binary
             Search binary files, but do not attempt to print them.

[MikePall]

Thanks for the investigation. I'll stay with the hardcoded way for OSX (it always has unwind tables), but switch to -qU for the benefit of the BSD users.