Fail to Build on AArch64 Device Due to Undefined _Float16 type

[1awrenceYang]

🐛 Describe the bug

Hi everyone, thanks for your effort on this issue.

I'm trying to build executorch on my OrangePi 5 Pro board equipped with an 8 core ARMv8 CPU, But I encountered a compile error. Here are steps to reproduce the error:

I clone and download submodules as required:

(executorch) orangepi@orangepi5pro:~/Desktop$ git clone -b release/0.4 https://github.com/pytorch/executorch.git
Cloning into 'executorch'...
remote: Enumerating objects: 150656, done.
remote: Counting objects: 100% (2392/2392), done.
remote: Compressing objects: 100% (898/898), done.
remote: Total 150656 (delta 1698), reused 2018 (delta 1452), pack-reused 148264 (from 1)
Receiving objects: 100% (150656/150656), 139.32 MiB | 37.85 MiB/s, done.
Resolving deltas: 100% (118236/118236), done.
(executorch) orangepi@orangepi5pro:~/Desktop$ cd executorch/
(executorch) orangepi@orangepi5pro:~/Desktop/executorch$ git submodule sync
(executorch) orangepi@orangepi5pro:~/Desktop/executorch$ git submodule update --init
Submodule 'backends/arm/third-party/ethos-u-core-driver' (https://review.mlplatform.org/ml/ethos-u/ethos-u-core-driver) registered for path 'backends/arm/third-party/ethos-u-core-driver'
Submodule 'backends/arm/third-party/serialization_lib' (https://review.mlplatform.org/tosa/serialization_lib) registered for path 'backends/arm/third-party/serialization_lib'
Submodule 'backends/cadence/hifi/third-party/nnlib/nnlib-hifi4' (https://github.com/foss-xtensa/nnlib-hifi4.git) registered for path 'backends/cadence/hifi/third-party/nnlib/nnlib-hifi4'
Submodule 'backends/vulkan/third-party/Vulkan-Headers' (https://github.com/KhronosGroup/Vulkan-Headers) registered for path 'backends/vulkan/third-party/Vulkan-Headers'
Submodule 'backends/vulkan/third-party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'backends/vulkan/third-party/VulkanMemoryAllocator'
Submodule 'backends/vulkan/third-party/volk' (https://github.com/zeux/volk) registered for path 'backends/vulkan/third-party/volk'
Submodule 'backends/xnnpack/third-party/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'backends/xnnpack/third-party/FP16'
Submodule 'backends/xnnpack/third-party/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'backends/xnnpack/third-party/FXdiv'
Submodule 'backends/xnnpack/third-party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'backends/xnnpack/third-party/XNNPACK'
Submodule 'backends/xnnpack/third-party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'backends/xnnpack/third-party/cpuinfo'
Submodule 'backends/xnnpack/third-party/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'backends/xnnpack/third-party/pthreadpool'
Submodule 'examples/third-party/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'examples/third-party/fbjni'
Submodule 'extension/llm/third-party/abseil-cpp' (https://github.com/abseil/abseil-cpp.git) registered for path 'extension/llm/third-party/abseil-cpp'
Submodule 'extension/llm/third-party/re2' (https://github.com/google/re2.git) registered for path 'extension/llm/third-party/re2'
Submodule 'extension/llm/third-party/sentencepiece' (https://github.com/google/sentencepiece.git) registered for path 'extension/llm/third-party/sentencepiece'
Submodule 'kernels/optimized/third-party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'kernels/optimized/third-party/eigen'
Submodule 'third-party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third-party/flatbuffers'
Submodule 'third-party/flatcc' (https://github.com/dvidelabs/flatcc.git) registered for path 'third-party/flatcc'
Submodule 'third-party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third-party/gflags'
Submodule 'third-party/googletest' (https://github.com/google/googletest.git) registered for path 'third-party/googletest'
Submodule 'third-party/ios-cmake' (https://github.com/leetal/ios-cmake) registered for path 'third-party/ios-cmake'
Submodule 'third-party/prelude' (https://github.com/facebook/buck2-prelude.git) registered for path 'third-party/prelude'
Submodule 'third-party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third-party/pybind11'
Cloning into '/home/orangepi/Desktop/executorch/backends/arm/third-party/ethos-u-core-driver'...
Cloning into '/home/orangepi/Desktop/executorch/backends/arm/third-party/serialization_lib'...
Cloning into '/home/orangepi/Desktop/executorch/backends/cadence/hifi/third-party/nnlib/nnlib-hifi4'...
Cloning into '/home/orangepi/Desktop/executorch/backends/vulkan/third-party/Vulkan-Headers'...
Cloning into '/home/orangepi/Desktop/executorch/backends/vulkan/third-party/VulkanMemoryAllocator'...
Cloning into '/home/orangepi/Desktop/executorch/backends/vulkan/third-party/volk'...
Cloning into '/home/orangepi/Desktop/executorch/backends/xnnpack/third-party/FP16'...
Cloning into '/home/orangepi/Desktop/executorch/backends/xnnpack/third-party/FXdiv'...
Cloning into '/home/orangepi/Desktop/executorch/backends/xnnpack/third-party/XNNPACK'...
Cloning into '/home/orangepi/Desktop/executorch/backends/xnnpack/third-party/cpuinfo'...
Cloning into '/home/orangepi/Desktop/executorch/backends/xnnpack/third-party/pthreadpool'...
Cloning into '/home/orangepi/Desktop/executorch/examples/third-party/fbjni'...
Cloning into '/home/orangepi/Desktop/executorch/extension/llm/third-party/abseil-cpp'...
Cloning into '/home/orangepi/Desktop/executorch/extension/llm/third-party/re2'...
Cloning into '/home/orangepi/Desktop/executorch/extension/llm/third-party/sentencepiece'...
Cloning into '/home/orangepi/Desktop/executorch/kernels/optimized/third-party/eigen'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/flatbuffers'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/flatcc'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/gflags'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/googletest'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/ios-cmake'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/prelude'...
Cloning into '/home/orangepi/Desktop/executorch/third-party/pybind11'...
Submodule path 'backends/arm/third-party/ethos-u-core-driver': checked out '90f9df900acdc0718ecd2dfdc53780664758dec5'
Submodule path 'backends/arm/third-party/serialization_lib': checked out '187af0d41fe75d08d2a7ec84c1b4d24b9b641ed2'
Submodule path 'backends/cadence/hifi/third-party/nnlib/nnlib-hifi4': checked out '6a9ea45e23ef591fe207442df33a5ebe88bbe8de'
Submodule path 'backends/vulkan/third-party/Vulkan-Headers': checked out '0c5928795a66e93f65e5e68a36d8daa79a209dc2'
Submodule path 'backends/vulkan/third-party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191'
Submodule path 'backends/vulkan/third-party/volk': checked out 'b3bc21e584f97400b6884cb2a541a56c6a5ddba3'
Submodule path 'backends/xnnpack/third-party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3'
Submodule path 'backends/xnnpack/third-party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1'
Submodule path 'backends/xnnpack/third-party/XNNPACK': checked out '87ee0b46b834f67bad9025d4a82ed5654f3403d3'
Submodule path 'backends/xnnpack/third-party/cpuinfo': checked out '16bfc1622c6902d6f91d316ec54894910c620325'
Submodule path 'backends/xnnpack/third-party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8'
Submodule path 'examples/third-party/fbjni': checked out '52a14f0daa889a20d8984798b8d96eb03cebd334'
Submodule path 'extension/llm/third-party/abseil-cpp': checked out 'eb852207758a773965301d0ae717e4235fc5301a'
Submodule path 'extension/llm/third-party/re2': checked out '6dcd83d60f7944926bfd308cc13979fc53dd69ca'
Submodule path 'extension/llm/third-party/sentencepiece': checked out '6225e08edb2577757163b3f5dbba4c0b670ef445'
Submodule path 'kernels/optimized/third-party/eigen': checked out 'a39ade4ccf99df845ec85c580fbbb324f71952fa'
Submodule path 'third-party/flatbuffers': checked out '595bf0007ab1929570c7671f091313c8fc20644e'
Submodule path 'third-party/flatcc': checked out '896db54787e8b730a6be482c69324751f3f5f117'
Submodule path 'third-party/gflags': checked out 'a738fdf9338412f83ab3f26f31ac11ed3f3ec4bd'
Submodule path 'third-party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
Submodule path 'third-party/ios-cmake': checked out '06465b27698424cf4a04a5ca4904d50a3c966c45'
Submodule path 'third-party/prelude': checked out '4e9e6d50b8b461564a7e351ff60b87fe59d7e53b'
Submodule path 'third-party/pybind11': checked out '8c7b8dd0ae74b36b7d42f77b0dd4096ebb7f4ab1'

I then start executing the installing_requirements.sh, then I got this compile error, they are all about some missing member 'y'

 /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:46:5: error: ‘_Float16’ does not name a type; did you mean ‘BFloat16’?
     46 |     _Float16 y;
        |     ^~~~~~~~
        |     BFloat16
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h: In constructor ‘executorch::runtime::etensor::Half::Half(float)’:
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:373:34: error: class ‘executorch::runtime::etensor::Half’ does not have any field named ‘y’
    373 | inline Half::Half(float value) : y(value) {}
        |                                  ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h: In member function ‘executorch::runtime::etensor::Half::operator float() const’:
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:382:17: error: ‘y’ was not declared in this scope; did you mean ‘yn’?
    382 |   return (float)y;
        |                 ^
        |                 yn
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h: In function ‘executorch::runtime::etensor::Half executorch::runtime::etensor::operator+(const executorch::runtime::etensor::Half&, const executorch::runtime::etensor::Half&)’:
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:9: error: ‘struct executorch::runtime::etensor::Half’ has no member named ‘y’
    397 |     ret.y = r;         \
        |         ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:402:3: note: in expansion of macro ‘return_half’
    402 |   return_half(a.y + b.y);
        |   ^~~~~~~~~~~
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:402:17: error: ‘const struct executorch::runtime::etensor::Half’ has no member named ‘y’
    402 |   return_half(a.y + b.y);
        |                 ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:13: note: in definition of macro ‘return_half’
    397 |     ret.y = r;         \
        |             ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:402:23: error: ‘const struct executorch::runtime::etensor::Half’ has no member named ‘y’
    402 |   return_half(a.y + b.y);
        |                       ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:13: note: in definition of macro ‘return_half’
    397 |     ret.y = r;         \
        |             ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h: In function ‘executorch::runtime::etensor::Half executorch::runtime::etensor::operator-(const executorch::runtime::etensor::Half&, const executorch::runtime::etensor::Half&)’:
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:9: error: ‘struct executorch::runtime::etensor::Half’ has no member named ‘y’
    397 |     ret.y = r;         \
        |         ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:406:3: note: in expansion of macro ‘return_half’
    406 |   return_half(a.y - b.y);
        |   ^~~~~~~~~~~
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:406:17: error: ‘const struct executorch::runtime::etensor::Half’ has no member named ‘y’
    406 |   return_half(a.y - b.y);
        |                 ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:13: note: in definition of macro ‘return_half’
    397 |     ret.y = r;         \
        |             ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:406:23: error: ‘const struct executorch::runtime::etensor::Half’ has no member named ‘y’
    406 |   return_half(a.y - b.y);
        |                       ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:13: note: in definition of macro ‘return_half’
    397 |     ret.y = r;         \
        |             ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h: In function ‘executorch::runtime::etensor::Half executorch::runtime::etensor::operator*(const executorch::runtime::etensor::Half&, const executorch::runtime::etensor::Half&)’:
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:9: error: ‘struct executorch::runtime::etensor::Half’ has no member named ‘y’
    397 |     ret.y = r;         \
        |         ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:411:3: note: in expansion of macro ‘return_half’
    411 |   return_half(a.y * b.y);
        |   ^~~~~~~~~~~
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:411:17: error: ‘const struct executorch::runtime::etensor::Half’ has no member named ‘y’
    411 |   return_half(a.y * b.y);
        |                 ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:13: note: in definition of macro ‘return_half’
    397 |     ret.y = r;         \
        |             ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:411:23: error: ‘const struct executorch::runtime::etensor::Half’ has no member named ‘y’
    411 |   return_half(a.y * b.y);
        |                       ^
  /home/orangepi/Desktop/executorch/../executorch/runtime/core/portable_type/half.h:397:13: note: in definition of macro ‘return_half’
    397 |     ret.y = r;         \
        |             ^
        ...... and more

I realize that this fault starts with the undefined _Float16. Because _Float16 is undefined and therefore member y is also undefined. How should I fix this? Any suggestions is greatly appreciated.

Versions

Here are my environment running the collect_env.py:

(executorch) orangepi@orangepi5pro:~/Desktop/executorch$ python collect_env.py
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
Collecting environment information...
PyTorch version: 2.5.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.5 LTS (aarch64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.31.0
Libc version: glibc-2.35

Python version: 3.10.0 (default, Mar  3 2022, 09:51:40) [GCC 10.2.0] (64-bit runtime)
Python platform: Linux-6.1.43-rockchip-rk3588-aarch64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       ARM
Model name:                      Cortex-A55
Model:                           0
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
Stepping:                        r2p0
CPU max MHz:                     1800.0000
CPU min MHz:                     408.0000
BogoMIPS:                        48.00
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
Model name:                      Cortex-A76
Model:                           0
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
Stepping:                        r4p0
CPU max MHz:                     2352.0000
CPU min MHz:                     408.0000
BogoMIPS:                        48.00
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
L1d cache:                       384 KiB (8 instances)
L1i cache:                       384 KiB (8 instances)
L2 cache:                        2.5 MiB (8 instances)
L3 cache:                        3 MiB (1 instance)
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Vulnerable: Unprivileged eBPF enabled
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.5.0
[pip3] torchaudio==2.5.0
[pip3] torchsr==1.0.4
[pip3] torchvision==0.20.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] torch                     2.5.0                    pypi_0    pypi
[conda] torchaudio                2.5.0                    pypi_0    pypi
[conda] torchsr                   1.0.4                    pypi_0    pypi
[conda] torchvision               0.20.0                   pypi_0    pypi

[GregoryComer]

@manuelcandales Could you take a look at this? Thanks!

[cbilgin]

Tagging @digantdesai who might be able to repro this as well.

[digantdesai]

Thanks @1awrenceYang for the detailed report.

For older Arm GCC (<13.1 IIRC) we need to use __fp16 and include <arm_fp16.h>, but for newer Arm GCC, _Float16 is available. Here is a small repro with your compiler failing but newer one working.

We can try and add support for the older gcc (as clang seems to have always supported this).

[1awrenceYang]

Thanks @1awrenceYang for the detailed report.

For older Arm GCC (<13.1 IIRC) we need to use __fp16 and include <arm_fp16.h>, but for newer Arm GCC, _Float16 is available. Here is a small repro with your compiler failing but newer one working.

We can try and add support for the older gcc (as clang seems to have always supported this).

Many thanks for @digantdesai 's constructive suggestion. After switching to ARM gcc 13 the problem is solved. I'm able to follow the start guide and get expected output now.

For anyone facing the same issue. The solution is updating your gcc to gcc 13+. On my Orange Pi 5 Pro with RK3588 ARM processor this works: sudo apt update && sudo apt install -y software-properties-common && sudo add-apt-repository ppa:ubuntu-toolchain-r/test && sudo apt install -y gcc-13 g++-13. After updating the gcc make sure to config the environment variable: export CC=gcc-13 && export CXX=g++-13. This should fix the problem.

I would like to express my gratitude again for your effort on this issue. I'm closing it as it's solved.

[BodhiHu]

upgrading the compiler to clang17 fixed for me

[spalatinate]

Hi @1awrenceYang - many thanks for your solution. Actually, I am working on a RPi5 and I encountered a similar solution. Unfortunately, I am getting the error message below. May I ask for a hint? I was strictly following your solution above. Many thanks in advance!

  CMake Error at /home/miniconda3/envs/executorch/lib/python3.10/site-packages/cmake/data/share/cmake-3.31/Modules/CMakeDetermineCCompiler.cmake:49 (message):
    Could not find compiler set in environment variable CC:

    gcc-13.
  Call Stack (most recent call first):
    CMakeLists.txt:45 (project)


  CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
  CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
  -- Configuring incomplete, errors occurred!
  error: command '/home/miniconda3/envs/bin/cmake' failed with exit code 1
  error: subprocess-exited-with-error
  
  × Building wheel for executorch (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/miniconda3/envs/bin/python /home/miniconda3/envs/executorch/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmptjr3ciog
  cwd: /home/llava/executorch
  Building wheel for executorch (pyproject.toml) ... error
  ERROR: Failed building wheel for executorch
Failed to build executorch
ERROR: Failed to build installable wheels for some pyproject.toml based projects (executorch)
Traceback (most recent call last):
  File "/home/llava/executorch/./install_requirements.py", line 198, in <module>
    subprocess.run(
  File "/home/miniconda3/envs/executorch/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/miniconda3/envs/executorch/bin/python', '-m', 'pip', 'install', '.', '--no-build-isolation', '-v', '--extra-index-url', 'https://download.pytorch.org/whl/test/cpu']' returned non-zero exit status 1.

[1awrenceYang]

Hi @spalatinate , From what I'm seeing the error message "Could not find compiler set in environment variable CC:" implies that your gcc13 is not properly installed. As this issue is closed. Please open a new issue with a complete description of your local environment so the community can look into your problem in detail. Thanks!