-fsycl-targets is order dependent

[fwyzard]

It looks like the behaviour of the -fsycl-targets flag is somewhat dependent on the order of the targets.

Passing -fsycl-targets=nvptx64-unknown-unknown-sycldevice,spir64-unknown-unknown-sycldevice results in a binary that can run on both backends, CUDA and OpenCL:

$ /data/user/fwyzard/sycl/build/bin/clang++ -O2 -g -Wno-unknown-cuda-version -fsycl -fsycl-targets=nvptx64-unknown-unknown-sycldevice,spir64-unknown-unknown-sycldevice --cuda-path=/usr/local/cuda simple-sycl-app.cpp -o simple-sycl-app

$ SYCL_BE=PI_OPENCL ./simple-sycl-app
Available SYCL devices:
  - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18
  - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711
  - Tesla K40c, driver version CUDA 10.20
  - SYCL host device, driver version 1.2

Running on SYCL device Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711
element 0 is 0
element 1 is 1
element 2 is 2
element 3 is 3
The results are correct!

$ SYCL_BE=PI_CUDA ./simple-sycl-app
Available SYCL devices:
  - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18
  - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711
  - Tesla K40c, driver version CUDA 10.20
  - SYCL host device, driver version 1.2

Running on SYCL device Tesla K40c, driver version CUDA 10.20
element 0 is 26
element 1 is 26
element 2 is 26
element 3 is 26
The results are correct!

Passing -fsycl-targets=spir64-unknown-unknown-sycldevice,nvptx64-unknown-unknown-sycldevice results in a binary that can run only on the OpenCL backend:

$ /data/user/fwyzard/sycl/build/bin/clang++ -O2 -g -Wno-unknown-cuda-version -fsycl -fsycl-targets=spir64-unknown-unknown-sycldevice,nvptx64-unknown-unknown-sycldevice --cuda-path=/usr/local/cuda simple-sycl-app.cpp -o simple-sycl-app

$ SYCL_BE=PI_OPENCL ./simple-sycl-app
Available SYCL devices:
  - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18
  - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711
  - Tesla K40c, driver version CUDA 10.20
  - SYCL host device, driver version 1.2

Running on SYCL device Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711
element 0 is 0
element 1 is 1
element 2 is 2
element 3 is 3
The results are correct!

$ SYCL_BE=PI_CUDA ./simple-sycl-app
Available SYCL devices:
  - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18
  - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711
  - Tesla K40c, driver version CUDA 10.20
  - SYCL host device, driver version 1.2

Running on SYCL device Tesla K40c, driver version CUDA 10.20
terminate called after throwing an instance of 'cl::sycl::feature_not_supported'
  what():  Online compilation is not supported in this context 0 (CL_SUCCESS)
Aborted (core dumped)

Running on Ubuntu 18.04 with the sycl branch as of 80b0306.

[fwyzard]

@krasznaa FYI

[fwyzard]

passing nvptx64 as the first target:

$ /data/user/fwyzard/sycl/build/bin/clang++ -O2 -g -Wno-unknown-cuda-version -fsycl -fsycl-targets=nvptx64-*-*-sycldevice,spir64-*-*-sycldevice --cuda-path=/usr/local/cuda-10.2 simple-sycl-app.cpp -o simple-sycl-app
$ ./simple-sycl-app --cuda
Available SYCL platforms:
  - Intel(R) OpenCL, driver version OpenCL 2.1 LINUX
    - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18

  - Intel(R) OpenCL HD Graphics, driver version OpenCL 2.1 
    - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711

  - NVIDIA CUDA, driver version CUDA 10.20
    - Tesla K40c, driver version CUDA 10.20

  - SYCL host platform, driver version 1.2
    - SYCL host device, driver version 1.2

Running on SYCL device Tesla K40c, driver version CUDA 10.20
The results are correct!

passing spir64 as the first target:

$ /data/user/fwyzard/sycl/build/bin/clang++ -O2 -g -Wno-unknown-cuda-version -fsycl -fsycl-targets=spir64-*-*-sycldevice,nvptx64-*-*-sycldevice --cuda-path=/usr/local/cuda-10.2 simple-sycl-app.cpp -o simple-sycl-app
$ ./simple-sycl-app --cuda
Available SYCL platforms:
  - Intel(R) OpenCL, driver version OpenCL 2.1 LINUX
    - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18

  - Intel(R) OpenCL HD Graphics, driver version OpenCL 2.1 
    - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711

  - NVIDIA CUDA, driver version CUDA 10.20
    - Tesla K40c, driver version CUDA 10.20

  - SYCL host platform, driver version 1.2
    - SYCL host device, driver version 1.2

Running on SYCL device Tesla K40c, driver version CUDA 10.20
pi_die: cuda_piProgramCreate not implemented
terminate called without an active exception
Aborted (core dumped)

passing spir64 as the first target, with #1223:

$ /data/user/fwyzard/sycl/build/bin/clang++ -O2 -g -Wno-unknown-cuda-version -fsycl -fsycl-targets=spir64-*-*-sycldevice,nvptx64-*-*-sycldevice --cuda-path=/usr/local/cuda-10.2 simple-sycl-app.cpp -o simple-sycl-app
$ ./simple-sycl-app --cuda
Available SYCL platforms:
  - Intel(R) OpenCL, driver version OpenCL 2.1 LINUX
    - Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, driver version 2020.9.1.0.18

  - Intel(R) OpenCL HD Graphics, driver version OpenCL 2.1 
    - Intel(R) Gen9 HD Graphics NEO, driver version 20.07.15711

  - NVIDIA CUDA, driver version CUDA 10.20
    - Tesla K40c, driver version CUDA 10.20

  - SYCL host platform, driver version 1.2
    - SYCL host device, driver version 1.2

Running on SYCL device Tesla K40c, driver version CUDA 10.20
The results are correct!