Failed to build C++ code for customized CUDA kernels

[hytu99]

Description

A build error occurs when attempting to compile the C++ code for CUDA inference. Below are the details of the issue.

Environment

• Docker Image: nvidia/cuda:12.1.0-cudnn8-devel-ubuntu20.04
• Base OS: Ubuntu 20.04.6 LTS
• GPU / Driver / CUDA: GeForce RTX 3090 / Driver Version: 535.129.03 / CUDA Version: 12.2
• Compiler & Version: g++ 9.4.0
• Build System: CMake 3.16.3 / Ninja 1.10.0

Steps to Reproduce

sudo apt-get install cmake g++ ninja-build
cd ../layers/extensions/inference/
pip install .

Error Message

The build fails with the following error:

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: inference_extensions_cuda
  Building wheel for inference_extensions_cuda (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [210 lines of output]
      running bdist_wheel
      running build
      running build_ext
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/utils/cpp_extension.py:426: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
        warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
      building 'inference_extensions_cuda' extension
      creating /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312
      Emitting ninja build file /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      [1/3] /usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/kernel.o.d -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/TH -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/envs/dcvc/include/python3.12 -c -c /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu -o /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math --extra-device-vectorization -arch=native -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=inference_extensions_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
      FAILED: /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/kernel.o
      /usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/kernel.o.d -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/TH -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/envs/dcvc/include/python3.12 -c -c /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu -o /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math --extra-device-vectorization -arch=native -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=inference_extensions_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(104): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(y_res, y_q, y_hat, s_hat, y, scales, means, mask,
                           ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(104): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(y_res, y_q, y_hat, s_hat, y, scales, means, mask,
                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(104): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(y_res, y_q, y_hat, s_hat, y, scales, means, mask,
                                                 ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(105): error: structured binding cannot be captured
                                                           static_cast<scalar_t>(force_zero_thres), N);
                                                                                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(108): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(y_res, y_q, y_hat, s_hat, y, scales, means, mask,
                           ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(108): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(y_res, y_q, y_hat, s_hat, y, scales, means, mask,
                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(108): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(y_res, y_q, y_hat, s_hat, y, scales, means, mask,
                                                 ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(109): error: structured binding cannot be captured
                                                           static_cast<scalar_t>(force_zero_thres), N);
                                                                                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(324): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, cond_out.value(), scales, scale_min, scale_max,
                           ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(324): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, cond_out.value(), scales, scale_min, scale_max,
                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(324): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, cond_out.value(), scales, scale_min, scale_max,
                                                 ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(325): error: structured binding cannot be captured
                                                           log_scale_min, log_step_recip, skip_thres, N);
                                                                                                      ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(328): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, nullptr, scales, scale_min, scale_max,
                           ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(328): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, nullptr, scales, scale_min, scale_max,
                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(328): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, nullptr, scales, scale_min, scale_max,
                                                 ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(329): error: structured binding cannot be captured
                                                           log_scale_min, log_step_recip, skip_thres, N);
                                                                                                      ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(392): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, cond_out.value(), symbols, scales,
                           ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(392): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, cond_out.value(), symbols, scales,
                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(392): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, cond_out.value(), symbols, scales,
                                                 ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(394): error: structured binding cannot be captured
                                                           log_step_recip, skip_thres, N);
                                                                                       ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(397): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, nullptr, symbols, scales, scale_min, scale_max,
                           ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(397): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, nullptr, symbols, scales, scale_min, scale_max,
                                    ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(397): error: structured binding cannot be captured
                        <<<gridDim, blockDim, 0, stream>>>(out, nullptr, symbols, scales, scale_min, scale_max,
                                                 ^

      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu(398): error: structured binding cannot be captured
                                                           log_scale_min, log_step_recip, skip_thres, N);
                                                                                                      ^

      24 errors detected in the compilation of "/aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/kernel.cu".
      [2/3] c++ -MMD -MF /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/impl.o.d -pthread -B /opt/conda/envs/dcvc/compiler_compat -fno-strict-overflow -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/conda/envs/dcvc/include -fPIC -O2 -isystem /opt/conda/envs/dcvc/include -fPIC -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/TH -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/envs/dcvc/include/python3.12 -c -c /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/impl.cpp -o /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/impl.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=inference_extensions_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
      [3/3] c++ -MMD -MF /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/bind.o.d -pthread -B /opt/conda/envs/dcvc/compiler_compat -fno-strict-overflow -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/conda/envs/dcvc/include -fPIC -O2 -isystem /opt/conda/envs/dcvc/include -fPIC -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/TH -I/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/envs/dcvc/include/python3.12 -c -c /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/bind.cpp -o /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/build/temp.linux-x86_64-cpython-312/bind.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=inference_extensions_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
      In file included from /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/csrc/Exceptions.h:12,
                       from /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/csrc/api/include/torch/python.h:11,
                       from /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/torch/extension.h:9,
                       from /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/def.h:4,
                       from /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/bind.cpp:4:
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h: In instantiation of ‘class pybind11::class_<DepthConvProxy>’:
      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/bind.cpp:23:51:   required from here
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h:1588:7: warning: ‘pybind11::class_<DepthConvProxy>’ declared with greater visibility than its base ‘pybind11::detail::generic_type’ [-Wattributes]
       1588 | class class_ : public detail::generic_type {
            |       ^~~~~~
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h: In instantiation of ‘class pybind11::class_<SubpelConv2xProxy>’:
      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/bind.cpp:31:57:   required from here
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h:1588:7: warning: ‘pybind11::class_<SubpelConv2xProxy>’ declared with greater visibility than its base ‘pybind11::detail::generic_type’ [-Wattributes]
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h: In instantiation of ‘pybind11::class_< <template-parameter-1-1>, <template-parameter-1-2> >::class_(pybind11::handle, const char*, const Extra& ...) [with Extra = {}; type_ = DepthConvProxy; options = {}]’:
      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/bind.cpp:23:51:   required from here
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h:1648:28: warning: ‘pybind11::class_< <template-parameter-1-1>, <template-parameter-1-2> >::class_(pybind11::handle, const char*, const Extra& ...) [with Extra = {}; type_ = DepthConvProxy; options = {}]::<lambda(pybind11::detail::internals&)>’ declared with greater visibility than the type of its field ‘pybind11::class_< <template-parameter-1-1>, <template-parameter-1-2> >::class_(pybind11::handle, const char*, const Extra& ...) [with Extra = {}; type_ = DepthConvProxy; options = {}]::<lambda(pybind11::detail::internals&)>::<record capture>’ [-Wattributes]
       1648 |             with_internals([&](internals &internals) {
            |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~
       1649 |                 auto &instances = record.module_local ? get_local_internals().registered_types_cpp
            |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       1650 |                                                       : internals.registered_types_cpp;
            |                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       1651 |                 instances[std::type_index(typeid(type_alias))]
            |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       1652 |                     = instances[std::type_index(typeid(type))];
            |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       1653 |             });
            |             ~
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h: In instantiation of ‘pybind11::class_< <template-parameter-1-1>, <template-parameter-1-2> >::class_(pybind11::handle, const char*, const Extra& ...) [with Extra = {}; type_ = SubpelConv2xProxy; options = {}]’:
      /aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/bind.cpp:31:57:   required from here
      /opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/include/pybind11/pybind11.h:1648:28: warning: ‘pybind11::class_< <template-parameter-1-1>, <template-parameter-1-2> >::class_(pybind11::handle, const char*, const Extra& ...) [with Extra = {}; type_ = SubpelConv2xProxy; options = {}]::<lambda(pybind11::detail::internals&)>’ declared with greater visibility than the type of its field ‘pybind11::class_< <template-parameter-1-1>, <template-parameter-1-2> >::class_(pybind11::handle, const char*, const Extra& ...) [with Extra = {}; type_ = SubpelConv2xProxy; options = {}]::<lambda(pybind11::detail::internals&)>::<record capture>’ [-Wattributes]
      ninja: build stopped: subcommand failed.
      Traceback (most recent call last):
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2104, in _run_ninja_build
          subprocess.run(
        File "/opt/conda/envs/dcvc/lib/python3.12/subprocess.py", line 573, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/aiarena/gpfs/codes2025/DCVC/src/layers/extensions/inference/setup.py", line 17, in <module>
          setup(
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 186, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
          dist.run_commands()
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 983, in run_commands
          self.run_command(cmd)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
          super().run_command(command)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
          cmd_obj.run()
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 369, in run
          self.run_command("build")
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
          self.distribution.run_command(command)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
          super().run_command(command)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
          cmd_obj.run()
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 136, in run
          self.run_command(cmd_name)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
          self.distribution.run_command(command)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/dist.py", line 999, in run_command
          super().run_command(command)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
          cmd_obj.run()
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
          _build_ext.run(self)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 365, in run
          self.build_extensions()
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 868, in build_extensions
          build_ext.build_extensions(self)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 481, in build_extensions
          self._build_extensions_serial()
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 507, in _build_extensions_serial
          self.build_extension(ext)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 264, in build_extension
          _build_ext.build_extension(self, ext)
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 562, in build_extension
          objects = self.compiler.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 681, in unix_wrap_ninja_compile
          _write_ninja_file_and_compile_objects(
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1784, in _write_ninja_file_and_compile_objects
          _run_ninja_build(
        File "/opt/conda/envs/dcvc/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2120, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for inference_extensions_cuda
  Running setup.py clean for inference_extensions_cuda
Failed to build inference_extensions_cuda
ERROR: Failed to build installable wheels for some pyproject.toml based projects (inference_extensions_cuda)

``` 🚀

[Serein810]

Hi!
Have you ever solved this problem? I met this problem just few days ago and I cant solve it. If you solve it would you please tell me the solution?
Thanks a lot.

[yaohualibin]

Would you please try a more recent version of pytorch and CUDA, such as pytorch-2.6.0 and CUDA-12.6?

[yaohualibin]

Did you successfully build the kernels using a more recent version of pytorch and CUDA?

[hytu99]

Did you successfully build the kernels using a more recent version of pytorch and CUDA?

No, I was not able to build the kernels successfully. The highest CUDA version supported by the machines available to me (driver version 535.129.03) is CUDA 12.2, which might not be compatible with the required setup.

[yaohualibin]

Then I think the building failure was casued by non-supported CUDA version. I will close the issue now. Please feel free to reopen it if you need further help.

[urbste]

I had the same issue. Probably due to CUDA version but you can fix it!

You need to manually unpack the tuples returned by the get_kernel_launch_info and get_kernel_launch_info_flatten functions. You must apply this change to every "dispatcher" function in the file that uses this pattern.

For example:
auto [blockDim, gridDim, stream, useVec, biasSafe, N, HW] = get_kernel_launch_info<vec_t>(y);

Needs to be changed to:

const auto launch_info = get_kernel_launch_info<vec_t>(y);
const dim3& blockDim = std::get<0>(launch_info);
const dim3& gridDim = std::get<1>(launch_info);
const at::cuda::CUDAStream& stream = std::get<2>(launch_info);
const bool useVec = std::get<3>(launch_info);
const bool biasSafe = std::get<4>(launch_info); // biasSafe is not used, can be omitted
const int N = std::get<5>(launch_info);
const int HW = std::get<6>(launch_info); // HW is not used, can be omitted

Either you do it manually or let your favorite AI Agent do it ;)

[LangdonLee]

kernel.txt
I've modified kernel.cu so it can be compiled with CUDA=12.2
You may download this txt file and paste into ./src/layers/extentions/inference/kernel.cu : )