Bootstrap problems when setting `#define MEMDEBUG` in `options.h`

[eschnett]

I am trying to track down a segfault, and am thus setting #define MEMDEBUG when building Julia from scratch. This leads to the following messages during bootstrap:

linalg.jl
Warning: New definition 
    convert could not show value of type Tuple at linalg/symmetric.jl:21
is ambiguous with: 
    convert could not show value of type Tuple at linalg/symmetric.jl:20.
To fix, define 
    convert could not show value of type Tuple
before the new definition.
Warning: New definition 
    convert could not show value of type Tuple at linalg/symmetric.jl:24
is ambiguous with: 
    convert could not show value of type Tuple at linalg/symmetric.jl:23.
To fix, define 
    convert could not show value of type Tuple
before the new definition.

This is version 7ca8cbdb7c066f3c6f34dd48307c835bc7ed1f19 of Julia, which is the current head of the master branch.

[eschnett]

The above is for OS X; on Linux, I see

[...]
simdloop.jl
Terminals.jl
LineEdit.jl

signal (11): Segmentation fault
julia_type_to_llvm at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/./cgutils.cpp:408
typed_load at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/./cgutils.cpp:889
emit_known_call at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2171
emit_call at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2283
emit_expr at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2749
emit_jlcall at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2251
emit_call at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2359
emit_expr at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2749
emit_jlcall at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2251
emit_call at /xfs1/eschnetter/compute/SIMFACTORY/julia-master/src/julia-327ebeebda87c9d078f50cdfd4ee182196a87b51/src/codegen.cpp:2359
[...]

[eschnett]

See also https://github.com/eschnett/julia/tree/enable-MEMDEBUG and https://travis-ci.org/eschnett/julia/builds/35789397.

[JeffBezanson]

Thank you. I am looking into this.

[ihnorton]

Here is a valgrind of bootstrapping (doesn't crash so maybe not so helpful):
https://gist.github.com/ihnorton/cd03e81e4f61c968d184

[JeffBezanson]

Was that run with MEMDEBUG?

[ihnorton]

Yes

[binarybana]

MEMDEBUG seems to work fine for me with release-0.3 branch if that datapoint helps anyone. Maybe a git bisect would help then?

@eschnett , for your original segfault, MEMDEBUG didn't help much in my experience for #8129 (sorry for the shameless plug, but I'm desperate over there!). So I wish you luck.

[eschnett]

@binarybana Travis says that release-0.3 with MEMDEBUG bootstraps but then fails the self-tests https://github.com/eschnett/julia/compare/release-0.3-enable-MEMDEBUG. Do you see the same?

[ihnorton]

@eschnett if you are trying to use this for debugging code outside of the system image, you might try working around this by building the system as normal and then rebuilding only in src/ with MEMDEBUG defined. This ultimately changes runtime behavior so should not impact the image code at all.

[eschnett]

Bisected the history. The offending merge commit is 0e55472. The previous commit 743af1a is fine.

I tested this on Linux, with this patch

perl -pi -e 's+//#define MEMDEBUG+#define MEMDEBUG+' src/options.h

and build options

BUILDOPTS='prefix=/xfs1/eschnetter/compute/julia VERBOSE=1 USE_BLAS64=0 USE_OPENBLAS=0 USE_SYSTEM_ARPACK=1 USE_SYSTEM_BLAS=1 USE_SYSTEM_FFTW=1 USE_SYSTEM_GMP=1 USE_SYSTEM_LAPACK=1 USE_SYSTEM_MPFR=1 USE_SYSTEM_PCRE=1 USE_SYSTEM_SUITESPARSE=1 USE_SYSTEM_ZLIB=1'

I ran make debug release, running the tests is not necessary. The symptoms are error messages of the kind

simdloop.jl
Terminals.jl
LineEdit.jl

signal (11): Segmentation fault
julia_type_to_llvm at /xfs1/eschnetter/compute/src/julias/julia/usr/bin/../lib/libjulia-debug.so (unknown line)
unknown function (ip: 41227722)
unknown function (ip: 41358682)
unknown function (ip: 41360716)

[binarybana]

Git bisect always feels a bit like warm sunshine and magic fairy dust when it works.

Oh, and I didn't run the tests when I had MEMDEBUG on, sorry about that.

[eschnett]

I said "I bisected", not "git bisected"... I used copy-and-past from a GUI to get the list of commit ids, and inspected the output visually since builds fail in so many different ways.

[eschnett]

@JeffBezanson: The failing commit merges your branch jb/size0stuff. I've looked at it, but I don't understand the details. Could you have a look whether you can reproduce the bootstrap failure with MEMDEBUG?

[ihnorton]

@eschnett this appears to be fixed now.