Bevy Colors V2

[viridia]

There's been some discussion about rethinking the way Bevy represents colors. This ticket is a brief summary of the issues; a formal specification will come later.

Statement of the problem

The current representation of Color is an enum which provides an enumeration of various color spaces (SRGB, HSL, etc.) Unfortunately, this flexibility comes at a cost:

• It's difficult to add additional color spaces without changing a lot of Bevy engine code.
• There are many cases where its more performant to have the color space is known at compile time. For example, when specifying things like lighting colors or shader params, supporting multiple color spaces incurs runtime overhead.
• Some operations are easier to do in some color spaces than others. For example, hue shift is trivial in HSL space, but more complex in RGB. Adding convenience methods for these kinds of operations introduces additional complexity for little apparent value.
  • Someone wanting a nice color gradient can easily convert their RGBA colors to Oklab or HSL and then call mix() on the result.
  • Similarly, if you want to do a hue shift, convert to HSL, modify the hue, and convert back. This is better than having Bevy try to implement hue shifts in RGB.
• Users would like color operations such as lerp or mix which produce different results for different color spaces. Moreover, it either requires that all of the inputs be in the same color space, or it must automatically convert the inputs to a common space. Both of these choices create additional complexity and runtime overhead.
• Having separate types for separate color spaces can help avoid footguns around forgetting to convert to the correct color space.

Proposed solution

A new bevy crate will be introduced, named bevy_color. This will have concrete types for several different color spaces:

• LinRgbaColor
• LinRgbColor (Note: it's an open question as to whether we need the non-alpha colors).
• SRgbaColor
• SRgbColor
• HslColor
• OklabColor
• And possibly others as needed.
• Bikeshedding on the names is permitted :)

The conversion between color spaces will be implemented via the From trait. These conversions will never fail; if a color cannot be represented in the target space, then a suitable, reasonably close value will be chosen. The only methods that return an error result will be deserialization / decoding methods.

In addition, for the Linear and SRGB spaces, there will be a .transmute_xxx() method to deal with the rare case where you have the wrong type encoded - i.e. the image decoder only produces SRGB types, but the actual image file is linear.

There will be traits which add additional color manipulation operations. Not all traits will be defined for every color space:

• Mix (linear interpolation)
• Add
• Multiply
• Darken
• Lighten
• Saturate
• Desaturate
• HexString (Convert to/from "#nnnnnn")
• CssString (Convert to/from "hsl(60, 100%, 50%)" for example)
• A11y (e.g. .is_readable(contrasting_color))

Because the existing Color class is used so widely, it will be retained for the next several Bevy releases to ease migration. It may eventually be replaced with a new class which allows multiple bevy_color colors to be represented in an enum.

To ease the pain of transition, API methods that accept a Color can be updated to accept an impl IntoLinRgba trait which automatically converts the argument to LinRgba.

To address the need for named standard colors (like Color::Red), a new namespace bevy_color::names will contain definitions for all the standard colors.

Alternative solutions

There has been proposals to integrate the palette crate, which is a very full-featured library for color representation and manipulation. However, a cursory look at this crate's source code suggests that it is over-engineered for our purpose. Bevy would be better served with a more lightweight solution that only includes the capabilities we are likely to actually need.

See also #1402

[hymm]

palette pr here #9698

Palette does give pretty much all the features asked for above. I think we should consider it as long as we can show that it's not bloating compile times.

[MarkusTheOrt]

I am very much in favor of reworking colors!

I was thinking about possible implementations and whilst it might not produce all the nicest of code, I think the ergonomics of using PhantomData should not be overlooked.

instead of having SRGBColor, or HSLColor we'd then have Color<SRGB>, Color<HSL>.
this would give us type hints at compile time whilst being a zero cost abstraction.

Color conversion can still be done using the From trait as the following is still valid code:

impl From<Color<SRGB>> for Color<HSL> {
    ...
}

[mockersf]

See also #4648

[viridia]

@MarkusTheOrt - I'm not sure what the advantage is of doing this, it doesn't seem any more or less ergonomic. Honestly, I really don't think we need to do anything clever here - the most straightforward and obvious approach will work well enough.

@hymm - I think this is one of those questions where we will have to take the temperature of the Bevy community and/or leadership. If they decide they want to support palette, that's fine, I can work with that. However, if for whatever reason they decide not to support palette, then I think my proposal provides the best alternative.

[viridia]

There's a couple of changes I would make to the proposal:

1. Drop the non-alpha types from the proposal.
2. Feedback from discord suggests the name "Color" is redundant, so you'd just have "LinearRgba", "SRgba", etc.

One nice thing about this proposal is that the work is very distributable - anyone can take a trait or color space and work on just that.

Also, as Cart has pointed out, all the algorithms are widely known, standardized, there's not much room for variation, we can cut & paste from other places, once it's done it's not likely to change much.

Open question: would there be much benefit to a SIMD color space, for example DeviceColorSIMD? Seems like this would only be useful for operations that manipulated large numbers of individual colors.

[JMS55]

Great link from @LPGhatguy https://developer.chrome.com/docs/css-ui/high-definition-css-color-guide#what_is_a_color_space

[viridia]

I have done a bit more research on this issue and have refined my ideas slightly. There are also some open questions.

Transmutes

Incorrect encoding: it sometimes happens, especially dealing with image files, where you have a color that would normally be sRGB but is in fact linear. For example, something like an illuminance map output from a renderer. So there is a need to be able to convert from one color space data type to another without actually transforming the color values.

Looking at the palette crate, I see this can be done by something like this:

// Transmute a linear color to SRGB
let rgba = SRgba.from_components(linear.to_components());

Conversely, you can force the conversion of the components using the same method + into:

// Transmute a linear color to SRGB
let rgba = SRgba.from_components(rgba.into::<LinearRgba>.to_components());

Polymorphism

For most use cases where you need to convert from one color space to another, you can do the conversions eagerly:

let srgba: SRgba = linear.into();

APIs that are meant to accept colors in multiple color spaces can take an Into parameter:

fn draw_rect(color: impl Into<LinearRgba>) {
    draw_rect_inner(color.into());
}

This makes draw_rect generic on the color space. It will convert the color to LinearRgba first and then call the inner function which is non-generic.

For things like bundle attributes (BackgroundColor) we can either convert the color to linear eagerly, or define some dyn trait like ToDeviceColor or possibly ToLinearRgba:

struct BackgroundColor {
    pub color: Box<dyn ToDeviceColor>,
};

This trait would have a single function, .to_device_color(), or possibly .to_linear_rgba() which converts the color from its current color space to linear RGBA. However, this approach has several downside, one of which is the need to box the color, the other is that there's no way to extract the color in its original color space if you need it for some reason.

For animating colors (examples such as CSS animations like hover states), ideally we would want the animation to be defined in a perceptual space during tweening - according to Raph Levien's post, Oklab works well for this purpose. This will require that we keep the control points of the animation (start and end) in Oklab, and convert the interpolated value to linear each frame. Bevy currently doesn't support such animations, but a UI framework is certainly going to want this.

This is assuming, of course, that the user will be satisfied with our choice of color space (there is precedent for this - in browsers, you don't get to choose what color space colors are lerped in). If the user really wants to have the choice to lerp in either RGB or Oklab space, then we need to store the start and endpoints in their original form, polymorphically. The starting and ending color have to be in the same color space, and it's hard to enforce this via static typing.

I'm hesitant to introduce a Color style enum because of its non-extensibility. If the user wants to work in XYZ color space, and Bevy does not provide it, this is not a problem for most things - the user can define their own XYZ struct type and implement the various traits we provide. But what they cannot do is add a new enum variant to Color without modifying Bevy itself.

The only other way to do runtime polymorphism is dyn traits, which does allow the user to extend the set of implementations - however, this requires that we know in advance all the possible methods that are likely to get called on the color value. Plus, you know, boxing.

Named colors

Currently there are about two dozen named colors (ALICE_BLUE and so on), which are all defined in sRGB. A question is whether users are going to want named colors in other color spaces - do we want named::linear::ALICE_BLUE or is it sufficient to have the user do the conversion themselves? This gets more interesting when we start thinking about the NONE / transparent color: if we're using pre-converted colors in rendering and we want to detect NONE, do we want a named::linear::NONE which is distinct from named::srgba::NONE?

[JMS55]

Imo we should remove the named colors anyways.

[viridia]

OK, I figured out some more.

The original impetus behind all this was color interpolation, which works a lot better if both the start and end colors are known to be in the same color space. For something like a CSS color transition, you can define a struct which does this:

/// Represents a range of colors that can be linearly interpolated, defined by a start and
/// end point which must be in the same color space.
pub struct ColorRange<T: Mix> {
    start: T,
    end: T,
}

impl<T> ColorRange<T>
where
    T: Mix,
{
    /// Get the color value at the given interpolation factor, which should be between 0.0
    /// and 1.0.
    pub fn at(&self, factor: f32) -> T {
        self.start.mix(&self.end, factor)
    }
}

ColorRange only requires that the color type implements the mix() method, which does the actual lerp.

This works great if the types of your colors are known statically. What if you really do need dynamic typing? In that case, we can define a dynamic trait which wraps a ColorRange:

trait AnyColorRange {
    fn at_linear(&self, factor: f32) -> LinearRgba;
}

The trait performs the interpolation and also converts the result to linear RGBA.

I'm pretty happy with this approach; it's pretty efficient, and while the dyn trait does require a box, it's generally only one box per animating widget, which is not a high price to pay.

[viridia]

Prototype library here: https://github.com/viridia/quill/blob/main/crates/bevy_color/src/lib.rs

[jnhyatt]

Looks like palette was ruled out for being too complex, has any thought been given to kolor? It seems to support the color spaces mentioned here, it's based on glam, and general seems fairly lightweight while still having most of what we want.

And tbh this feels close enough to what we want that adding another Bevy crate seems like overkill to me. If nothing else, we could use kolor as a base and add functionality to it somewhere in Bevy as needed.