A Distinctive Uniform Colour Space: The Colour Science Foundations of ACES

The Colour Science Foundations of ACES

ACES colour science gets mentioned all the time, and it seems to be a very high-end processing pipeline. From a colour science point of view, what ACES actually does is mainly embodied in the output transform (ODT). ACES provides a unified, reversible output transform for rendering scene-referred linear light onto a specified display. This output transform decides whether the image gets a good base rendering and a good starting point for grading, and it ensures results are as consistent as possible across displays with different brightness and gamut. If you switch between ACES versions, the difference in look also comes mainly from the output transform.

In ACES 2.0, the output transform is based on a colour space whose internal variable name is JMh, and clearly it has a deep connection to the CIECAM family of colour appearance models and colour spaces. In the ACES documentation, the space is described as a simplified Hellwig 2022 colour appearance model. The Hellwig 2022 model is itself a modification of the CAM family, claiming better colour appearance prediction. After the Hellwig modifications and the ACES modifications, the JMh space used by ACES 2.0 has actually drifted quite far from the original CIECAM.

So why call this JMh space the colour science foundation of ACES? Even though the output transform algorithm is rather complex, it still resembles the mappings found elsewhere in colour management: convert the colour into a uniform colour space, then modify lightness and chroma while holding hue, in order to change the image style or map it into the display gamut.

The hue linearity, uniformity and lightness prediction of JMh itself determine the overall hue linearity and image style of ACES, and these effects are not easily influenced by other grading tools.

The Structure of JMh

Since this is a new uniform colour space, I wanted a deeper understanding of it, for example:

whether it can be used on its own
what has changed compared with Hellwig 2022 and CIECAM16
how it performs as a uniform colour space
whether there are special treatments specific to ACES

Next I will break the space down in depth and try to answer these questions.

LMS and Chromatic Adaptation

Like most colour spaces, although the input is XYZ, the first step is to convert it into a space representing cone response, here called LMS. Working in cone response space generally gives better hue linearity and brightness prediction. This step is done with a 3x3 matrix.

Hellwig 2022 and CIECAM16 use the same cone space, and the matrix is called Matrix16. If you treat this matrix as a kind of XYZ to RGB matrix, then cone response can be seen as a sort of three-primary space, and it can be plotted on the CIE 1931 or 1976 u’v’ diagram.

ACES uses a huge intermediate space: ACES 2065-1, also called AP0. On the chromaticity diagram, some regions of this space are even larger than the RGB space derived from Matrix16. Mathematically this means that after AP0 is converted into LMS, part of the values become negative. That is very unfriendly for later processing. These are all physically non-existent colours and in theory should never appear, but after the output transform they become physically displayable colours, and these negative values produce colour discontinuities or even errors.

ACES CAM16 JMh on CIE 1976 UCS diagram

To deal with this, ACES moved the colour coordinate corresponding to the S-cone so that it covers AP0 as much as possible. This change happened around 2023, and if you are interested you can dig through the ACES development record and the meeting notes. The change may also improve hue linearity a little, and it makes the blue region slightly brighter after the output transform compared with the original Matrix16.

Effect on Blue due to change the matrix 16 from CIECAM16

After embedding ST2086 metadata this still looks odd on most devices, and it seems only the Macbook Pro and XDR look good. Use a wide gamut HDR display with at least 10x headroom for the best result.

This is a white plaster ornament lit with coloured RGB, shot through S-Gamut3.Cine-SLog3, then output through the ACES 2.0 output transform onto a BT.2020 1000 nits display. The only difference between the two images is the blue primary coordinate of the LMS space used by the output transform JMh.

Actually the AP0 out-of-gamut problem mentioned above may never be hit at all at the moment. The current version of the output transform clips from AP0 to AP1 in its first step, and AP1 is a much smaller gamut, so the difference may only be in the brightness of blue.

After converting to LMS, chromatic adaptation and alignment to the equal-energy white are applied. This step simply divides LMS by the ratio of the ACES white point to the equal-energy white, which amounts to complete adaptation.

Removing the Offset

Uniform colour spaces usually apply a non-linear function to the adapted LMS, simulating the non-linear post-adaptation of the visual system. These spaces all use the CIECAM sigmoid form, and ACES removes the 0.1 offset term at the end.

f (v) = \frac{400 (F_{L} v)^{0.42}}{27.13 + (F_{L} v)^{0.42}}

After post-adaptation, the three cone responses are combined in certain proportions into an achromatic response. Hellwig and CIECAM16 include a constant term, while ACES does not.

A = 2 R_{a}^{'} + G_{a}^{'} + 0.05 B_{a}^{'} - 0.305

Changing the LMS space and removing the offset term may help hue linearity somewhat. For example, on the Hung & Berns data, ACES JMh performs a bit better than CAM16-UCS, though in the blue region it still does not match spaces such as Jzazbz.

ACES JMh Hue Linearity

Simplified Colourfulness

The chroma and colourfulness calculation in CIECAM16 is very complex and involves a lot of intermediate quantities. Hellwig 2022 and ACES JMh simplify this into the traditional opponent structure: two chromatic axes (red–green, yellow–blue) are combined from the post-adapted cone responses, and then converted to polar coordinates to obtain chroma and hue.

[\begin{array}{ccc} 2 & 1 & \frac{1}{20} \\ 1 & - \frac{12}{11} & \frac{1}{11} \\ \frac{1}{9} & \frac{1}{9} & - \frac{2}{9} \end{array}]

This matrix is left-multiplied onto the column vector of post-adapted cone responses. The first row combines them into the achromatic response, and the last two rows combine them into the two chromatic axes. Multiplying the chroma in polar coordinates by a fixed scaling factor gives the ACES colourfulness M.

Hellwig 2022 colourfulness needs one more term, the eccentricity factor $e_{t}$ , which is a hue dependent quantity used to scale chroma with hue in order to improve chroma uniformity across different hues. In the ACES output transform this job is moved inside the output transform and is not carried by the uniformity of JMh.

The Final Form of JMh

After all these simplifications and improvements, it is closer to Jzazbz, sUCS or IPT than to a colour appearance model or a CIECAM analogue.

They all include the following steps:

Convert XYZ into cone response space (first 3x3 matrix)
Non-linear mapping of cone response (non-linear function)
Opponent combination of the post-adapted cone responses (second 3x3 matrix)

ACES has an extra post-processing step for lightness, similar to Jzazbz.

J = 100.0 {(\frac{A}{A_{w}})}^{c z}

This step is also not really necessary inside the output transform. The brightness mapping of the output transform still needs to convert back to linear light, and after mapping it steps back to lightness J.

A Colour Appearance Model?

ACES calls this a colour appearance model, but in practice this JMh has no consideration of viewing conditions at all.

Although the process uses surround, $L_{A}$ and $Y_{b}$ , quantities that represent viewing conditions, they are all hard-coded. The forward and inverse transforms use the same set, and they cannot be specified from outside (unless you completely rewrite it by hand). So they do not really serve to represent viewing conditions.

From this angle it is more like a uniform colour space with some optimisations for ACES, and the ACES output transform in turn contains some hard-coded quantities matched to it, such as the chroma coefficient that varies with hue. So using it on its own, or simply swapping in another space for ACES, does not guarantee the result.

Hellwig 2022: L. Hellwig and M. D. Fairchild, “Brightness, lightness, colorfulness, and chroma in CIECAM02 and CAM16,” Color Res. Appl., vol. 47, no. 5, pp. 1083–1095, Oct. 2022, doi: 10.1002/col.22792.
CIECAM16: C. Li et al., “Comprehensive color solutions: CAM16, CAT16, and CAM16-UCS,” Color Res. Appl., vol. 42, no. 6, pp. 703–718, Dec. 2017, doi: 10.1002/col.22131.

As for the complete form of ACES JMh, in the original CTL it has been turned into very C style code, so it no longer corresponds step by step to the colour science style description above. The reader will need to break it down and rewrite it themselves.