#[repr(C)] C-like enums and out of range values

[nikomatsakis]

@sdroege asked me the following on IRC, and I .. wasn't sure of the answer:

nmatsakis: if i have a repr(C) enum with values A=1, B=2. and a C function with that enum as return type returns "3", is that leading us to undefined behaviour already? is it guaranteed that converting the enum to the corresponding integer type will give "3" and nothing bad happens, and the only possible bad thing that could happen is that if there is a exhaustive match (for the compiler) on the enum... that it's actually not exhaustive (because 3) and from there on things go wrong?

I wasn't sure what I thought the answer should be.

[arielb1]

With the current implementation this is UB - we emit !range metadata on loads.

I think that if we want to allow "invalid" values in enums, we need to think about it more - we e.g. probably want to allow things like comparisons.

[le-jzr]

From the programmer's (and FFI bindings) perspective, it would be ideal if C-like enums with explicit #[repr(...)] were considered valid for the entire range of the underlying int* type. It would be basically saying "the enum is allowed to have values that don't have name assigned". However, exhaustive matching would not be possible for such enums.

[RalfJung]

C++ seems to treat out-of-rage casts to enum as "unspecified values", which means after the cast it can have any valid value of the target range. Casting back to int does not have to yield the same result, it could return anything. I couldn't find the relevant part in the C standard.

[le-jzr]

It says in Annex I of N1570 (C11) that an implementation may generate warnings when

A value is given to an object of an enumerated type other than by assignment of an
enumeration constant that is a member of that type, or an enumeration object that has
the same type, or the value of a function that returns the same enumerated type
(6.7.2.2).

I couldn't find any other mention anywhere, and I checked the lists of implementation-defined and undefined behaviors.

[strega-nil]

@RalfJung incorrect - http://eel.is/c++draft/expr.static.cast#10

A value of integral or enumeration type can be explicitly converted to a complete enumeration type. The value is unchanged if the original value is within the range of the enumeration values ([dcl.enum]). Otherwise, the behavior is undefined.

http://eel.is/c++draft/dcl.enum#8

For an enumeration whose underlying type is fixed, the values of the enumeration are the values of the underlying type. Otherwise, for an enumeration where e[min] is the smallest enumerator and e[max] is the largest, the values of the enumeration are the values in the range b[min] to b[max], defined as follows: Let K be 1 for a two's complement representation and 0 for a ones' complement or sign-magnitude representation. b[max] is the smallest value greater than or equal to max(|e[min]| − K, |e[max]|) and equal to 2M − 1, where M is a non-negative integer. b[min] is zero if e[min] is non-negative and −(b[max] + K) otherwise. The size of the smallest bit-field large enough to hold all the values of the enumeration type is max(M, 1) if b[min] is zero and M + 1 otherwise. It is possible to define an enumeration that has values not defined by any of its enumerators. If the enumerator-list is empty, the values of the enumeration are as if the enumeration had a single enumerator with value 0.

(basically, this all says that the range of an enum is enough to hold all of the variants or'd together, for bitflags)

[RalfJung]

I stand corrected. And it seems the rules changed, because the one I quoted above still says out-of-range is unspecified value, not UB. That was C++14, your link seems to be a draft of C++17. Thanks!

[strega-nil]

@RalfJung it looks like that was changed with CWG1766. Seems it was to make it non-constexpr?

[sdroege]

AFAIU in C it's not undefined behaviour (or implementation-defined behaviour) to use values that are in range of the underlying integer type but not part of the enum. It basically is just syntactic sugar around integers and giving names to specific ones. Compiler warnings for using values outside the range of the enum were only added relatively recently (few years ago) too.
I guess it would be useful if repr(C) enums could have the same behaviour, it makes interfacing with C libraries a bit more convenient (if you have to otherwise use integers and explicitly convert between those and the enum), or safer (if you still use repr(C) enums but the C library could give you values outside the range of the enum defined in Rust, e.g. because a newer version added a new value in a backwards compatible way).

For C++ the story seems to be a bit different as @ubsan is quoting above.

[le-jzr]

It's also worth noting that enums are often not even used in C, due to their abysmal benefit. Instead a lot of enumerations are just a bunch of #define constants.

[arielb1]

@sdroege

I consider the Rust FFI equivalent of a C enum to be a newtype integer with a set of constants. All sorts of "semi-invalid" values are basically armed bombs waiting to UB up your program, to be touched very carefully, but with C-style enums, out-of-range values are not "invalid" in any way or form - using an "old" header with a "new" library is a to-be-explicitly-supported usecase.

[RalfJung]

I guess one question is whether repr should really have such a fundamental effect on the type (giving up on exhaustiveness) -- that goes beyond just deciding how values are represented in memory. Maybe there should instead just be a macro to define a newtype + constants like @arielb1 mentioned.

[le-jzr]

Or it could piggyback on the non-exhaustive enum RFC that's currently (hopefully) in the process of being accepted. We could define that #[repr(C/iXX/uXX)] + non-exhaustive => well defined for all values of the underlying​ type.

[sdroege]

I consider the Rust FFI equivalent of a C enum to be a newtype integer with a set of constants.

What's the exact meaning of repr(C) for an enum then, if not that it is "like an enum in C and compatible with that over FFI boundaries"?

[nagisa]

@sdroege as repr in the attribute implies, it only affects the representation in memory, not semantics. This is true for all items which admit a repr(C) attribute.