[offload] `libomptarget :: amdgcn-amd-amdhsa` tests are failing on Gentoo

[mgorny]

While I've been working on standalone build, I've been hitting a bunch of AMDGPU-related test failures. Curious enough, I hit the same failures when I do a regular build. Reproduced against 4ad8b26 most recently, but I've seen it with 19.1.4 too. Note that I've had to apply #118807 for offload to build.

My commands are:

cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS='clang;lld' -DLLVM_ENABLE_RUNTIMES='openmp;offload' -DCMAKE_BUILD_TYPE=MinSizeRel -DLLVM_CCACHE_BUILD=ON  -DFFI_INCLUDE_DIR="/usr/lib64/libffi/include" -DLLVM_TARGETS_TO_BUILD='AMDGPU;NVPTX;X86' -DCLANG_DEFAULT_LINKER=lld
ninja
ninja check-runtimes

which seems to roughly match what the CI is doing.

An example failure looks like this:

FAIL: libomptarget :: amdgcn-amd-amdhsa :: api/assert.c (395 of 1433)
******************** TEST 'libomptarget :: amdgcn-amd-amdhsa :: api/assert.c' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
/home/mgorny/git/llvm-project/build/./bin/clang -fopenmp    -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L /home/mgorny/git/llvm-project/build/./lib -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src  -nogpulib -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -Wl,-rpath,/home/mgorny/git/llvm-project/build/./lib  -fopenmp-targets=amdgcn-amd-amdhsa /home/mgorny/git/llvm-project/offload/test/api/assert.c -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a && /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp | /home/mgorny/git/llvm-project/build/./bin/FileCheck /home/mgorny/git/llvm-project/offload/test/api/assert.c
# executed command: /home/mgorny/git/llvm-project/build/./bin/clang -fopenmp -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L /home/mgorny/git/llvm-project/build/./lib -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -nogpulib -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -Wl,-rpath,/home/mgorny/git/llvm-project/build/./lib -fopenmp-targets=amdgcn-amd-amdhsa /home/mgorny/git/llvm-project/offload/test/api/assert.c -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a
# .---command stderr------------
# | ld.lld: error: undefined symbol: __kmpc_target_init
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | 
# | ld.lld: error: undefined symbol: __kmpc_target_deinit
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | 
# | ld.lld: error: undefined symbol: __assert_fail
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | clang: error: ld.lld command failed with exit code 1 (use -v to see invocation)
# | /home/mgorny/git/llvm-project/build/bin/clang-linker-wrapper: error: 'clang' failed
# | clang: error: linker command failed with exit code 1 (use -v to see invocation)
# `-----------------------------
# error: command failed with exit status: 1

--

A quick attempt at debugging suggests clang-linker-wrapper is not passing any libraries when linking:

$ /home/mgorny/git/llvm-project/build/./bin/clang -fopenmp -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L /home/mgorny/git/llvm-project/build/./lib -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -nogpulib -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -Wl,-rpath,/home/mgorny/git/llvm-project/build/./lib -fopenmp-targets=amdgcn-amd-amdhsa /home/mgorny/git/llvm-project/offload/test/api/assert.c -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a -v
clang version 20.0.0git (git@github.com:llvm/llvm-project.git 4ad8b2640eaa732da1a4c2429bcac26df3fe943f)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/mgorny/git/llvm-project/build/bin
Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/14
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
 "/home/mgorny/git/llvm-project/build/bin/clang-20" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -emit-llvm-uselists -dumpdir /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name assert.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/mgorny/git/llvm-project/build -v -fcoverage-compilation-dir=/home/mgorny/git/llvm-project/build -resource-dir /home/mgorny/git/llvm-project/build/lib/clang/20 -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fopenmp -nogpulib -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -disable-llvm-passes -fopenmp-targets=amdgcn-amd-amdhsa -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/assert-95f5c4.bc -x c /home/mgorny/git/llvm-project/offload/test/api/assert.c
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/home/mgorny/git/llvm-project/build/lib/clang/20/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/mgorny/git/llvm-project/offload/test
 /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src
 /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers
 /home/mgorny/git/llvm-project/build/lib/clang/20/include
 /usr/local/include
 /usr/include
End of search list.
 "/home/mgorny/git/llvm-project/build/bin/clang-20" -cc1 -triple amdgcn-amd-amdhsa -aux-triple x86_64-unknown-linux-gnu -Werror=atomic-alignment -emit-llvm-bc -emit-llvm-uselists -dumpdir /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name assert.c -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu gfx1012 -debugger-tuning=gdb -fdebug-compilation-dir=/home/mgorny/git/llvm-project/build -v -resource-dir /home/mgorny/git/llvm-project/build/lib/clang/20 -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include/openmp_wrappers -include __clang_openmp_device_functions.h -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fvisibility=protected -fopenmp -nogpulib -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -fopenmp-is-target-device -fopenmp-host-ir-file-path /tmp/assert-95f5c4.bc -faddrsig -o /tmp/assert-ea06cf.bc -x c /home/mgorny/git/llvm-project/offload/test/api/assert.c
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/home/mgorny/git/llvm-project/build/lib/clang/20/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/usr/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/home/mgorny/git/llvm-project/build/lib/clang/20/include"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/mgorny/git/llvm-project/offload/test
 /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src
 /home/mgorny/git/llvm-project/build/lib/clang/20/include/openmp_wrappers
 /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers
 /home/mgorny/git/llvm-project/build/lib/clang/20/include
 /usr/local/include
 /usr/include
End of search list.
 "/home/mgorny/git/llvm-project/build/bin/clang-offload-packager" -o /tmp/assert-0f1543.out --image=file=/tmp/assert-ea06cf.bc,triple=amdgcn-amd-amdhsa,arch=gfx1012,kind=openmp
 "/home/mgorny/git/llvm-project/build/bin/clang-20" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -dumpdir /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name assert.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/mgorny/git/llvm-project/build -v -fcoverage-compilation-dir=/home/mgorny/git/llvm-project/build -resource-dir /home/mgorny/git/llvm-project/build/lib/clang/20 -ferror-limit 19 -fopenmp -nogpulib -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -fembed-offload-object=/tmp/assert-0f1543.out -fopenmp-targets=amdgcn-amd-amdhsa -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/assert-75251f.o -x ir /tmp/assert-95f5c4.bc
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target x86_64-unknown-linux-gnu
 "/home/mgorny/git/llvm-project/build/bin/clang-linker-wrapper" --host-triple=x86_64-unknown-linux-gnu --wrapper-verbose --linker-path=/home/mgorny/git/llvm-project/build/bin/ld.lld --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/Scrt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/crtbeginS.o -L/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L/home/mgorny/git/llvm-project/build/./lib -L/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L/usr/lib/gcc/x86_64-pc-linux-gnu/14 -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/lib -L/lib -L/usr/lib -rpath /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -rpath /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -rpath /home/mgorny/git/llvm-project/build/./lib /tmp/assert-75251f.o /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a -lomp -lomptarget -L/home/mgorny/git/llvm-project/build/lib -lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/14/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/crtn.o -verbose
 "/home/mgorny/git/llvm-project/build/bin/clang" --no-default-config -o /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img --target=amdgcn-amd-amdhsa -mcpu=gfx1012 -O2 -flto -Wl,--no-undefined /tmp/assert-75251f-amdgcn-amd-amdhsa-gfx1012-8c8d61.o -v
clang version 20.0.0git (git@github.com:llvm/llvm-project.git 4ad8b2640eaa732da1a4c2429bcac26df3fe943f)
Target: amdgcn-amd-amdhsa
Thread model: posix
InstalledDir: /home/mgorny/git/llvm-project/build/bin
 "/home/mgorny/git/llvm-project/build/bin/ld.lld" --no-undefined -shared --no-undefined /tmp/assert-75251f-amdgcn-amd-amdhsa-gfx1012-8c8d61.o -plugin-opt=mcpu=gfx1012 -plugin-opt=O2 --lto-CGO2 -o /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img
ld.lld: error: undefined symbol: __kmpc_target_init
>>> referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
>>> referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)

ld.lld: error: undefined symbol: __kmpc_target_deinit
>>> referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
>>> referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)

ld.lld: error: undefined symbol: __assert_fail
>>> referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
>>> referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
clang: error: ld.lld command failed with exit code 1 (use -v to see invocation)
/home/mgorny/git/llvm-project/build/bin/clang-linker-wrapper: error: 'clang' failed
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Full build and test log (6.5M after unpacking): log.txt.gz

[llvmbot]

@llvm/issue-subscribers-offload

Author: Michał Górny (mgorny)

While I've been working on standalone build, I've been hitting a bunch of AMDGPU-related test failures. Curious enough, I hit the same failures when I do a regular build. Reproduced against 4ad8b26 most recently, but I've seen it with 19.1.4 too. Note that I've had to apply #118807 for offload to build.

My commands are:

cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS='clang;lld' -DLLVM_ENABLE_RUNTIMES='openmp;offload' -DCMAKE_BUILD_TYPE=MinSizeRel -DLLVM_CCACHE_BUILD=ON  -DFFI_INCLUDE_DIR="/usr/lib64/libffi/include" -DLLVM_TARGETS_TO_BUILD='AMDGPU;NVPTX;X86' -DCLANG_DEFAULT_LINKER=lld
ninja
ninja check-runtimes

which seems to roughly match what the CI is doing.

An example failure looks like this:

FAIL: libomptarget :: amdgcn-amd-amdhsa :: api/assert.c (395 of 1433)
******************** TEST 'libomptarget :: amdgcn-amd-amdhsa :: api/assert.c' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
/home/mgorny/git/llvm-project/build/./bin/clang -fopenmp    -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L /home/mgorny/git/llvm-project/build/./lib -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src  -nogpulib -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -Wl,-rpath,/home/mgorny/git/llvm-project/build/./lib  -fopenmp-targets=amdgcn-amd-amdhsa /home/mgorny/git/llvm-project/offload/test/api/assert.c -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a && /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp | /home/mgorny/git/llvm-project/build/./bin/FileCheck /home/mgorny/git/llvm-project/offload/test/api/assert.c
# executed command: /home/mgorny/git/llvm-project/build/./bin/clang -fopenmp -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L /home/mgorny/git/llvm-project/build/./lib -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -nogpulib -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -Wl,-rpath,/home/mgorny/git/llvm-project/build/./lib -fopenmp-targets=amdgcn-amd-amdhsa /home/mgorny/git/llvm-project/offload/test/api/assert.c -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a
# .---command stderr------------
# | ld.lld: error: undefined symbol: __kmpc_target_init
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | 
# | ld.lld: error: undefined symbol: __kmpc_target_deinit
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | 
# | ld.lld: error: undefined symbol: __assert_fail
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | >>> referenced by /tmp/lit-tmp-wxf1utdf/assert.c.tmp.amdgcn.gfx1012-0994c1.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
# | clang: error: ld.lld command failed with exit code 1 (use -v to see invocation)
# | /home/mgorny/git/llvm-project/build/bin/clang-linker-wrapper: error: 'clang' failed
# | clang: error: linker command failed with exit code 1 (use -v to see invocation)
# `-----------------------------
# error: command failed with exit status: 1

--

A quick attempt at debugging suggests clang-linker-wrapper is not passing any libraries when linking:

$ /home/mgorny/git/llvm-project/build/./bin/clang -fopenmp -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L /home/mgorny/git/llvm-project/build/./lib -L /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -nogpulib -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -Wl,-rpath,/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -Wl,-rpath,/home/mgorny/git/llvm-project/build/./lib -fopenmp-targets=amdgcn-amd-amdhsa /home/mgorny/git/llvm-project/offload/test/api/assert.c -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a -v
clang version 20.0.0git (git@<!-- -->github.com:llvm/llvm-project.git 4ad8b2640eaa732da1a4c2429bcac26df3fe943f)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/mgorny/git/llvm-project/build/bin
Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/14
Candidate multilib: .;@<!-- -->m64
Candidate multilib: 32;@<!-- -->m32
Candidate multilib: x32;@<!-- -->mx32
Selected multilib: .;@<!-- -->m64
 "/home/mgorny/git/llvm-project/build/bin/clang-20" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -emit-llvm-uselists -dumpdir /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name assert.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/mgorny/git/llvm-project/build -v -fcoverage-compilation-dir=/home/mgorny/git/llvm-project/build -resource-dir /home/mgorny/git/llvm-project/build/lib/clang/20 -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fopenmp -nogpulib -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -disable-llvm-passes -fopenmp-targets=amdgcn-amd-amdhsa -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/assert-95f5c4.bc -x c /home/mgorny/git/llvm-project/offload/test/api/assert.c
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/home/mgorny/git/llvm-project/build/lib/clang/20/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include &lt;...&gt; search starts here:
 /home/mgorny/git/llvm-project/offload/test
 /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src
 /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers
 /home/mgorny/git/llvm-project/build/lib/clang/20/include
 /usr/local/include
 /usr/include
End of search list.
 "/home/mgorny/git/llvm-project/build/bin/clang-20" -cc1 -triple amdgcn-amd-amdhsa -aux-triple x86_64-unknown-linux-gnu -Werror=atomic-alignment -emit-llvm-bc -emit-llvm-uselists -dumpdir /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name assert.c -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu gfx1012 -debugger-tuning=gdb -fdebug-compilation-dir=/home/mgorny/git/llvm-project/build -v -resource-dir /home/mgorny/git/llvm-project/build/lib/clang/20 -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include/openmp_wrappers -include __clang_openmp_device_functions.h -I /home/mgorny/git/llvm-project/offload/test -I /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /home/mgorny/git/llvm-project/build/lib/clang/20/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fvisibility=protected -fopenmp -nogpulib -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -fopenmp-is-target-device -fopenmp-host-ir-file-path /tmp/assert-95f5c4.bc -faddrsig -o /tmp/assert-ea06cf.bc -x c /home/mgorny/git/llvm-project/offload/test/api/assert.c
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/home/mgorny/git/llvm-project/build/lib/clang/20/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/usr/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/home/mgorny/git/llvm-project/build/lib/clang/20/include"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include &lt;...&gt; search starts here:
 /home/mgorny/git/llvm-project/offload/test
 /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src
 /home/mgorny/git/llvm-project/build/lib/clang/20/include/openmp_wrappers
 /home/mgorny/git/llvm-project/build/lib/clang/20/include/llvm_libc_wrappers
 /home/mgorny/git/llvm-project/build/lib/clang/20/include
 /usr/local/include
 /usr/include
End of search list.
 "/home/mgorny/git/llvm-project/build/bin/clang-offload-packager" -o /tmp/assert-0f1543.out --image=file=/tmp/assert-ea06cf.bc,triple=amdgcn-amd-amdhsa,arch=gfx1012,kind=openmp
 "/home/mgorny/git/llvm-project/build/bin/clang-20" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -dumpdir /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name assert.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/mgorny/git/llvm-project/build -v -fcoverage-compilation-dir=/home/mgorny/git/llvm-project/build -resource-dir /home/mgorny/git/llvm-project/build/lib/clang/20 -ferror-limit 19 -fopenmp -nogpulib -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -fembed-offload-object=/tmp/assert-0f1543.out -fopenmp-targets=amdgcn-amd-amdhsa -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/assert-75251f.o -x ir /tmp/assert-95f5c4.bc
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target x86_64-unknown-linux-gnu
 "/home/mgorny/git/llvm-project/build/bin/clang-linker-wrapper" --host-triple=x86_64-unknown-linux-gnu --wrapper-verbose --linker-path=/home/mgorny/git/llvm-project/build/bin/ld.lld --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload/test/amdgcn-amd-amdhsa/api/Output/assert.c.tmp /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/Scrt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/crtbeginS.o -L/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -L/home/mgorny/git/llvm-project/build/./lib -L/home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -L/usr/lib/gcc/x86_64-pc-linux-gnu/14 -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/lib -L/lib -L/usr/lib -rpath /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/offload -rpath /home/mgorny/git/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -rpath /home/mgorny/git/llvm-project/build/./lib /tmp/assert-75251f.o /home/mgorny/git/llvm-project/build/./lib/libomptarget.devicertl.a -lomp -lomptarget -L/home/mgorny/git/llvm-project/build/lib -lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-pc-linux-gnu/14/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../lib64/crtn.o -verbose
 "/home/mgorny/git/llvm-project/build/bin/clang" --no-default-config -o /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img --target=amdgcn-amd-amdhsa -mcpu=gfx1012 -O2 -flto -Wl,--no-undefined /tmp/assert-75251f-amdgcn-amd-amdhsa-gfx1012-8c8d61.o -v
clang version 20.0.0git (git@<!-- -->github.com:llvm/llvm-project.git 4ad8b2640eaa732da1a4c2429bcac26df3fe943f)
Target: amdgcn-amd-amdhsa
Thread model: posix
InstalledDir: /home/mgorny/git/llvm-project/build/bin
 "/home/mgorny/git/llvm-project/build/bin/ld.lld" --no-undefined -shared --no-undefined /tmp/assert-75251f-amdgcn-amd-amdhsa-gfx1012-8c8d61.o -plugin-opt=mcpu=gfx1012 -plugin-opt=O2 --lto-CGO2 -o /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img
ld.lld: error: undefined symbol: __kmpc_target_init
&gt;&gt;&gt; referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
&gt;&gt;&gt; referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)

ld.lld: error: undefined symbol: __kmpc_target_deinit
&gt;&gt;&gt; referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
&gt;&gt;&gt; referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)

ld.lld: error: undefined symbol: __assert_fail
&gt;&gt;&gt; referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
&gt;&gt;&gt; referenced by /tmp/assert.c.tmp.amdgcn.gfx1012-3a6ddc.img.lto.o:(__omp_offloading_31_4bb9d7e_main_l9)
clang: error: ld.lld command failed with exit code 1 (use -v to see invocation)
/home/mgorny/git/llvm-project/build/bin/clang-linker-wrapper: error: 'clang' failed
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Full build and test log (6.5M after unpacking): log.txt.gz

[jhuber6]

I'm guessing you have an AMD GPU w/ HSA installed to run these tests in the first place? The way it's supposed to work is that it picks up the build from inside the build-tree ${LLVM_LIBRARY_INTDIR}/libomptarget.devicertl.a I believe. Can you confirm whether that file exists in your build?

[mgorny]

I'm guessing you have an AMD GPU w/ HSA installed to run these tests in the first place?

I haven't used HSA before, so I'm not 100% sure if my setup is correct but I don't think it's even getting that far.

I do have:

$ ls /usr/lib64/libhsa*
/usr/lib64/libhsakmt.so    /usr/lib64/libhsakmt.so.1.0.6   /usr/lib64/libhsa-runtime64.so.1
/usr/lib64/libhsakmt.so.1  /usr/lib64/libhsa-runtime64.so  /usr/lib64/libhsa-runtime64.so.1.13.0

I've got a Navi 14 card, and amdgpu-arch reports gfx1012.

Can you confirm whether that file exists in your build?

That's from the build directory:

$ find -name 'libomptarget*.a'
./lib/libomptarget.rtl.amdgpu.a
./lib/libomptarget.rtl.cuda.a
./lib/libomptarget.rtl.host.a
./lib/libomptarget.devicertl.a

Oh sorry, I should have said that all x86-64 tests pass.

[jhuber6]

Hm, unsure why it's not picking it up then. It should be in the lit Python config that it looks at the INTDIR set in the test config. If this is the standalone, maybe that's not being set? It's the LLVM int dir I belive.

[mgorny]

It's a regular LLVM+Clang+LLD+runtimes build, I've listed the commands above. Lit is definitely passing the files over to clang, and clang is passing them to clang-linker-wrapper — but clang-linker-wrapper doesn't pass them through.

[jhuber6]

It's a regular LLVM+Clang+LLD+runtimes build, I've listed the commands above. Lit is definitely passing the files over to clang, and clang is passing them to clang-linker-wrapper — but clang-linker-wrapper doesn't pass them through.

Can you do llvm-objdump --offloading on the libomptarget.devicertl.a?

[mgorny]

$ ./bin/llvm-objdump --offloading lib/libomptarget.devicertl.a 

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx700.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx700
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx701.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx701
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx801.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx801
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx803.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx803
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx900.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx900
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx902.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx902
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx906.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx906
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx908.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx908
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx90a.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx90a
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx90c.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx90c
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx940.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx940
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx941.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx941
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx942.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx942
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx950.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx950
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1010.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1010
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1030.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1030
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1031.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1031
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1032.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1032
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1033.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1033
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1034.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1034
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1035.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1035
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1036.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1036
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1100.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1100
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1101.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1101
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1102.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1102
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1103.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1103
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1150.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1150
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1151.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1151
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1152.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1152
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-amdgpu-gfx1153.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            gfx1153
triple          amdgcn-amd-amdhsa
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_35.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_35
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_37.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_37
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_50.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_50
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_52.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_52
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_53.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_53
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_60.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_60
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_61.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_61
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_62.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_62
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_70.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_70
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_72.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_72
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_75.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_75
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_80.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_80
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_86.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_86
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_87.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_87
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_89.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_89
triple          nvptx64-nvidia-cuda
producer        openmp

lib/libomptarget.devicertl.a(devicertl-nvptx-sm_90.o):	file format elf64-x86-64

OFFLOADING IMAGE [0]:
kind            llvm ir
arch            sm_90
triple          nvptx64-nvidia-cuda
producer        openmp

So I guess it doesn't include my model?

[jhuber6]

Seems so, this is why this method of building one for each runtime is incredibly stupid. I want to rip all this out and build it the same way I build libc, but that's a lot of work. Set LIBOMPTARGET_DEVICE_ARCHITECTURES=gfx1012 for the personal fix, add it to https://github.com/llvm/llvm-project/blob/main/offload/DeviceRTL/CMakeLists.txt#L45 to solve this, or move to how I build my runtimes to actually solve it.

[mgorny]

Well, good news: adding it indeed makes most of the tests pass.

Bad news: one of the remaining tests crashed my GPU.

[jhuber6]

Well, good news: adding it indeed makes most of the tests pass.

Bad news: one of the remaining tests crashed my GPU.

Sounds about right, it deadlocks on mine about 80% of the time.