OSC segfaults, v2.x

[artpol84]

Today I noticed a segfault in our MTT:

# OSU MPI_Get Bandwidth Test v5.3.2
# Window creation: MPI_Win_allocate
# Synchronization: MPI_Win_flush
# Size      Bandwidth (MB/s)[boo13:12788] *** Process received signal ***
[boo13:12788] Signal: Segmentation fault (11)
[boo13:12788] Signal code: Address not mapped (1)
[boo13:12788] Failing at address: (nil)
[boo13:12788] [ 0] /usr/lib64/libpthread.so.0(+0xf100)[0x7ffff7608100]
[boo13:12788] [ 1] <mtt-base>/install/lib/openmpi/mca_osc_rdma.so(ompi_osc_rdma_unlock_atomic+0x1f9)[0x7fffe88639c9]
[boo13:12788] [ 2] <mtt-base>/install/lib/libmpi.so.20(PMPI_Win_unlock+0x1b)[0x7ffff7875f2b]
[boo13:12788] [ 3] <mtt-base>/tests/osu_micro_benchmark/osu-micro-benchmarks-5.3.2/mpi/one-sided/osu_get_bw[0x401f25]
[boo13:12788] [ 4] <mtt-base>/tests/osu_micro_benchmark/osu-micro-benchmarks-5.3.2/mpi/one-sided/osu_get_bw[0x4017e5]
[boo13:12788] [ 5] /usr/lib64/libc.so.6(__libc_start_main+0xf5)[0x7ffff7258b15]
[boo13:12788] [ 6] <mtt-base>/tests/osu_micro_benchmark/osu-micro-benchmarks-5.3.2/mpi/one-sided/osu_get_bw[0x401851]
[boo13:12788] *** End of error message ***srun: error: boo13: task 0: Segmentation fault

It was running fine for at least a couple of weeks before. This might be related to recently merged #3045.

[artpol84]

@karasevb, FIY

[hjelmn]

Hmm. And master is ok?

[artpol84]

We don't test it

[artpol84]

And this problem is 100% reproducible. I see it in all runs now.

[artpol84]

OMPI config is pretty basic - no Mellanox components are in this build.
But we use external pmix-v1.2, hwloc and libevent.
Also we use --with-platform=contrib/platform/mellanox/optimized but no Mellanox components are found.

[hjelmn]

I think I have an idea as to where the problem is. All my testing is done on a platform that has both fetching and non-fetching atomics. libibverbs only provides fetching. Will see if I can trigger the problem by disabling non-fetching atomics in btl/ugni.