Constraints for ABI

[UebelAndre]

Would it be possible to introduce @platforms//abi which contains constraints for common ABI definitions of platform triples?

In rules_rust we currently have issues with users wanting to target platforms with shared CPU and system and unique ABI but are unable to do so as there's no constraints uniquely identifying them. Users are forced to define custom constraints and redefine toolchains to get around errors caused by this ambiguity. I think it would be very beneficial to introduce ABI constraints here so that rules_rust and other rules can share constraints, enabling rule maintainers to appropriately constrain toolchains so users only have to setup platform definitions.

[hlopko]

Some old but related doc on the topic of more constraint settings: docs.google.com/document/d/1CgU-GKocMAfsUSI3bbGZ0YRkOWczitIoKs29x3zR914/edit#heading=h.3fbh1otqm5sz

[bsilver8192]

ARM ABIs are really complicated... I'm not sure how to fit all of this into constraints. I'm sure representing all of this complexity is a bad idea, but I don't have much input into where to draw the line.

I've been pretty happy defining my own constraint_setting that just enumerates my hardware platforms. The only place I find standard constraints helpful is using other people's BUILD files, which only needs granularity to the extent those BUILD files do different things. Most of this information is only needed by the toolchain itself, but I have seen different assembly files to choose between which do need more of this to work for any platform.

Some platforms I've targeted with Bazel which I think are common:

• Cortex-M0+ without FPU (small slow microcontroller) bare metal
• Cortex-M4 with FPU (faster microcontroller) bare metal. Note that the FPU is optional with this core, but all the ones I've seen have an FPU.
• Debian armhf (hf means FPU registers in the calling convention, Debian has some other variants too, but I haven't used them)
• Raspbian / Raspberry Pi OS (Debian armhf, but with flags to build for older processors)
• Debian armhf, plus all instructions supported by Cortex-A9
• Debian armhf, plus all instructions supported by Cortex-A53. I've run code compiled this way on both Debian arm64 (via multiarch) and armhf userspaces.
• Yocto-based Linux distribution on Cortex-A53 (I think the FPU is required on A53, mine has one and uses it in the calling convention)

Some less common platforms I've targeted with Bazel:

• Cortex-R5F with FPU (kind of like Cortex-A, but no MMU) with FreeRTOS
• Cortex-M4 with FPU and various RTOS
• Yocto-based Linux distribution on Cortex-A9 with an FPU, but with soft-float calling conventions

Some notes on aarch32 (ARMv6/ARMv7, or ARMv8 in 32-bit mode) variants I've dealt with:

• gnueabi and gnueabihf are the most common variants in the triples. gnueabihf corresponds to -mfloat-abi=hard for GCC, -mfloat-abi=soft and -mfloat-abi=softfp are the other choices which both have the same ABI but differ in whether floating point instructions are used within functions. AAPCS and EABI are two different documents that define parts of it. Object files are ELF and C++ code follows the Itanium ABI for both of them.
• -marm vs -mthumb compile functions with different instruction sets which are both aarch32. I think all EABI triples can freely call between functions compiled with each mode, which is called interworking (other ABIs can disable interworking). Most of the ways to call a function (but not all) in aarch32 will automatically switch between thumb and arm mode based on the low bit of the address, and the toolchain can set things up so that works everywhere with no additional cost beyond making sure to call functions with one of the instructions that does the switching.
• There's also OABI (old ABI), but I've never used that. This might be old enough to not worry about it.
• It's common to target a wide variety of ARM cores which have different instruction sets. There's both the choice of which core, and most of the cores have various optional instructions. This means it's pretty common to have different toolchains with the same triple that generate code compatible with different processors. Raspbian is a common example of this: it uses the same triple as Debian armhf but generates code compatible with some older processors.
• Different enum sizes are even more common with ARM than x86-land. ARM has officially documented variants for both 4-byte enums and sized-to-fit. Those make for really subtle ABI differences...
• https://bugs.llvm.org/show_bug.cgi?id=27455 is an even more subtle incompatibility I ran into where GCC and Clang are using different ABIs, despite trying to be compatible.
• Some of these variants are encoded in special sections in the ELF files so linking together incompatible ones will fail, others aren't.
• Some ARM toolchains come with a variety of precompiled libc/libgcc/etc for many of these variants.

[UebelAndre]

In hopes of moving the conversation along I've opened #39 to provide a concrete example of what I'm hoping for.

[aiuto]

IMO, there is no winning in trying to precisely define ARM platforms in this repository. Virtually every device is its own platform, since there can be so much customization of what is on the die.

[UebelAndre]

For what it's worth, the only 3 values I'm really hoping to represent are gnu, musl, and msvc which would help tremendously with producing docker images containing Rust or Go binaries. I'm not too familiar with all the existing ABI variants but are those considered base values (thinking of things like gnueabi)?

[aiuto]

I would rather go small first, with very precise definitions of what each constant means.
So what would those three things mean? I take ABI to strictly mean the calling convention between separately compiled modules. So, gnu is probably better called itaniam-C++. That should match clang too, so using the generic name is fair.

How is musl as an API different from gnu? Aren't we getting at features of the library, rather than the binary interface? Unless we are talking about different musls.
It is also not clear what msvc means as an ABI standard. The calling conventions have changed in different VS versions, and the windows APIs have changed as well.

Also, by being strict about calling conventions, I am excluding anything like linker format, and features of the standard library. Those should be an orthogonal space. It's not clear from the issue description if the need is to cover all of those topics or just the ABI.

@UebelAndre, it would help to see examples of what the rust teams are actually trying to do. Can you point to any of the platform definitions they are building? Or, maybe some definitive docs on rust/c++ interoperability.

[bsilver8192]

I would rather go small first, with very precise definitions of what each constant means.

I agree with these goals. I would like to add "no names that are prone to assuming an incorrect meaning" and "obvious nesting or mutual exclusion between all categories".

For example the current decision between //cpu:arm (all ARM? all aarch32? non-thumb aarch32?) vs //cpu:armv7 vs several sets of specific ARM cores denoted by their instruction set variants is really confusing. And is //cpu:arm64_32 aarch32-on-arm64 (code can use all the registers, but they're all caller-save and the ABI doesn't change) or is it the aarch64 version of x32? Even as somebody who knows what most of this means, I have no idea which of these are supposed to apply to my platforms.

I've got tons of specific things to bring up in the interest of avoiding something which confuses people thinking about just one of them. Also it'd be good to avoid precluding clean solutions to the rest of it in the future, but that might be hard.

So what would those three things mean? I take ABI to strictly mean the calling convention between separately compiled modules. So, gnu is probably better called itaniam-C++. That should match clang too, so using the generic name is fair.

Actually, coming from managing many C/C++ toolchains, gnu strikes me as the most problematic one on that list. For some triples, that means GNU OABI (which is ancient), vs gnueabi would be the GNU EABI that's used on all modern ARM. For other triples, on CPUs/platforms/etc which never used OABI, gnu means GNU EABI. Keep in mind that there might be some platforms with non-GNU EABI around too. Even if we document a consistent meaning for it within the BUILD file, people are going to misuse it because they assume it's obvious what it means (it's in my triple, that must be the one I want).

The itanium C++ ABI defines how C++ is implemented on top of common ELF ABIs. The name is because it was first defined for Itanium, but it's since become the de facto standard on all the common platforms (aarch32, aarch64, x86, amd64 are the ones I work with). But C++ ABI compatibility is also affected by (and this is just on Linux with libstdc++/libc++, Microsoft basically just breaks compatibility periodically instead):

• ELF symbol versioning (does your libstdc++ have it enabled? If so, what versions does it have?)
• C++ name mangling versioning (there's some compiler extensions for this, with complex choices around what your libstdc++ supports and which set you build against with the same header files)
• Different RTTIs which can make two modules that both work with the same C++ standard library not actually work when run together (because they end up with different std::string basically)

Separate from the mess of C++, there's also things like:

• enum sizes (-fshort-enums)
• relative function call offset ranges (-mlong-calls)
• whether complex math compiler builtin functions use the normal hard-float ABI or a special one (https://bugs.llvm.org/show_bug.cgi?id=27455)
• enabling exceptions and RTTI
• whether SIMD registers are callee-save

They do fit under "separately compiled modules need to handle them in compatible ways", but I don't think it's manageable in a centralized list like this... I think managing any of this with bazel platforms is a low priority.

How is musl as an API different from gnu? Aren't we getting at features of the library, rather than the binary interface? Unless we are talking about different musls. It is also not clear what msvc means as an ABI standard. The calling conventions have changed in different VS versions, and the windows APIs have changed as well.

Things like the interface to the dynamic linker and the implementation of errno depend on which libc. Rust also cares about the stack unwinding support. TLS implementation can also be different for some libc.

@UebelAndre, it would help to see examples of what the rust teams are actually trying to do. Can you point to any of the platform definitions they are building? Or, maybe some definitive docs on rust/c++ interoperability.

I believe the goal is to enable a platform to select a unique target from all the ones Rust supports at https://doc.rust-lang.org/nightly/rustc/platform-support.html, so that rules_rust can generate toolchains for all of them and get the correct one picked based on the platform.

[UebelAndre]

@UebelAndre, it would help to see examples of what the rust teams are actually trying to do. Can you point to any of the platform definitions they are building? Or, maybe some definitive docs on rust/c++ interoperability.

@aiuto I've created bazelbuild/rules_rust#1270