Re: [Gimp-developer] [Gimp-user] Time to fork BABL and GEGL

[Elle Stone <ellestone ninedegreesbelow com>]

On 11/18/2014 03:05 AM, Michael Henning wrote
> Now, please explain this to me with a straight answer: Why is it so
> insanely important to know what color space an operation happens in,
> in a situation where it*by definition*  does not matter, that you are
> willing to waste hours of your time and hours of developers' time
> arguing about it?

As usual I've written too many words and yes I've covered some of the 
points before. But this time there's an outline and an organized 
presentation.

We've agreed that for chromaticity dependent RGB editing operations ("CD 
ops", for short), editing in UserRGB versus unbounded sRGB will produce 
different results (except in the limiting case where UserRGB is sRGB), 
and so the correct thing to do is perform the edits using UserRGB 
chromaticities.

We've also agreed that for chromaticity independent RGB editing 
operations ("CI ops", for short), by definition the same colors (as 
located in the XYZ reference color space) are obtained regardless of the 
chromaticities that are used to encode the data.

Nonetheless CI ops should be done using UserRGB chromaticities.

Summary: Performing CI ops using sRGB chromaticities requires a whole 
lot of otherwise unnecessary implementation details and also requires 
identifying which operations really are CI. Proposed CIELAB workarounds 
won't work.


Outline:

1. Editing in alternate versions of GIMP that do and don't convert to 
unbounded sRGB for CI ops

2. Precision and CPU considerations

3. The onerous task of deciding which ops are CI ops

4. Dealing with Histograms, FG/BG Colors, Color Picker, and Gradients

5. CIELAB as a replacement for displaying and picking RGB colors: not a 
good idea

6. Replacing chromaticity dependent RGB ops with CIELAB ops: not a good idea

7. Moving past unbounded sRGB to HDR scene-referred editing


1. Editing in alternate versions of GIMP that do and don't convert to 
unbounded sRGB for CI ops:

Assume Person A is using a version of GIMP that uses UserRGB for all 
ops. Person A performs a CI op involving one or more image layers, 
followed by a CD op.

Assume Person B is using a version of GIMP that converts the image 
layer(s) to unbounded sRGB for all CI ops. Person B performs the same CI 
op as Person A, and then the image layer(s) are converted back to 
UserRGB for the CD op.

GIMP is a display-referred image editor and so all editing operations 
are presumed to be working on RGB channel values that are clipped to fit 
within the UserRGB channel data range 0.0 to 1.0.

When shooting raw, it's incredibly easy to record colors that exceed the 
very small bounded sRGB color gamut. Accordingly, converting the image 
from UserRGB to unbounded sRGB for CI ops will require unclipping all 
the CI ops. And upon conversion back to UserRGB, the unbounded sRGB 
colors will need to be clipped to the UserRGB color gamut.

If the CI op in question is a layer blend mode, then in reality to get 
the same UserRGB channel values as Person A, Person B will need to make 
a "New from Visible" layer from the blended layers, and then convert 
back to UserRGB.

Of course babl/GEGL/GIMP code can be added that makes this "New from 
Visible" layer be created automatically. And so you can consider this 
new code to be an implementation detail.

But now Person B is forced to deal with an additional layer in the stack 
that Person A doesn't have to deal with. But presumably a way can be 
coded up to make this "New from Visible" layer invisible to the user.

Throw a mask on the top blended layer. Now every time Person B modifies 
the mask, a new conversion to unbounded sRGB and a new "New from 
Visible" layer will need to be made before the layers are converted back 
to UserRGB.

Person A, of course, doesn't need a "New from Visible" layer.

If UserRGB is used for both CD and CI ops, there is no need to worry 
dealing with CI layer blend modes or unclipping CI ops and then clipping 
the unbounded results to fit within the UserRGB color gamut.


2. Precision and CPU considerations:

If CI ops are performed using sRGB chromaticities and CD ops are 
performed using UserRGB chromaticities, then every time a CI op is 
followed by a CD op, or vice versa, a conversion between UserRGB and 
sRGB will be required.

Let's make the assumption that babl/GEGL/GIMP throughput precision is 
equal to theoretical floating point precision (does anyone really think 
this is the case?). Even so, CPU time is required to perform conversions 
between UserRGB and unbounded sRGB.

Pippin has been working very hard to shave seconds off of various 
editing operations. It seems at best questionable to code up 
CPU-consuming conversions between UserRGB and unbounded sRGB only to get 
the same colors that UserRGB would have produced. So it's better to use 
UserRGB for both CD and CI ops.


3. The onerous task of deciding which ops are CI ops:

When performed on linearized RGB data, the list of chromaticity 
dependent editing operations is long, comprising over half of the 
roughly 80 GIMP UI operations that I checked.

When performed on perceptually uniform RGB data, the list of CD ops is 
much, much longer.

If the decision is made to perform CI editing operations in the 
unbounded sRGB color space, then the developers have to take on the 
responsibility of making sure that they don't make mistakes about which 
ops are CI and which are CD. Even if no official list is made, in effect 
the devs will be tasked with "making a list and checking it twice", 
which is fine if you are Santa Claus and have a lot of elvish help.

Performing all CI ops using the UserRGB chromaticities already produces 
correct results. Using UserRGB for CI ops never runs the risk of 
producing incorrect results because a developer didn't realize that 
"operation X" really was a CD op rather than a CI op.

So it seems to me that developer time is better spent on priorities 
other than figuring out which ops are CI and which are CD. If you always 
use UserRGB, the question never arises as to whether an editing op is CI 
or CD.


4. Dealing with Histograms, FG/BG Colors, Color Picker, and Gradients:

Let's assume the design decision is made to convert the image from 
UserRGB to unbounded sRGB for all CI ops.

Either the developers will need to code up histograms (Levels and Curves 
histograms as well as Info/Histogram) that show the out of gamut 
unbounded sRGB channel values  (currently everything is clipped to the 
0.0 to 1.0 range).

Or else the histogram displays will need to be shown using UserRGB 
channel values, even when the image layer is encoded using the sRGB 
chromaticities.

Similar considerations apply for selecting FG/BG Colors.

As the user really cannot be expected to understand the editing 
implications of those out of gamut sRGB-encoded channel values, the 
better course is to always display UserRGB-encoded channel values even 
when the image layer has been re-encoded using sRGB chromaticities.

The same is true for colors displayed by the Color Picker. To show why, 
consider the following:

On the one hand, everyone knows that in the ProPhotoRGB color space, 
ProPhotoRGB's reddest red is (1.0000, 0.0000, 0.0000).

On the other hand, GIMP users really cannot be expected to know that 
when encoded using linear gamma sRGB, (2.0344, -0.2288, -0.0085) is the 
same as ProPhotoRGB's reddest red.

Nor should users be expected to know that when encoded using 
perceptually uniform sRGB, (1.3633, 2.9569, -0.1104) is the same as 
ProPhotoRGB's reddest red. When a user color picks an RGB color, the 
user expects that color to be expressed as a *User*RGB color, not an 
unbounded sRGB color.


5. CIELAB as a replacement for displaying and picking RGB colors: not a 
good idea

A proposed alternative for picking and displaying unbounded sRGB-encoded 
colors is displaying colors in the CIELAB reference color space. In 
practice, LAB colors would be an invaluable *addition* to Color Picker 
and FB/BG Colors, especially if the boundary between real and imaginary 
colors could also be displayed.

But displaying CIELAB channel values is completely inadequate as a 
*replacement* for displaying UserRGB channel values because users 
sometimes really do need to know the actual UserRGB channel values.

For example, CIELAB colors are not going to help if UserRGB is AdobeRGB 
and the user wants to draw a linear gamma gradient from AdobeRGB reddest 
red to AdobeRGB greenest green.

For another example, if the user needs to know whether the Blue channel 
value is less than the Green channel value, then again the equivalent 
CIELAB values are useless.

Dismissing these cases as "corner cases" just won't fly. These "CIELAB 
won't work" examples represent critically important use cases for RGB 
image editing (at least for anyone who has moved past the "push the 
slider and see what happens" approach to image editing).


6. Replacing chromaticity dependent RGB ops with CIELAB ops: not a good idea

CIELAB is a wonderful *adjunct* to RGB image editing. But there is no 
case where a CD op OR a CI op should be *replaced* by a CIELAB op. The 
data coming off the camera sensor really is RGB data, not CIELAB data.

Operating on linearized RGB data allows photometrically 
("radiometrically") correct editing. BY DESIGN, CIELAB is not a 
photometrically correct color space:

* Color correction in CIELAB is useful, but it can't substitute for RGB 
color correction.

* Gradients drawn in CIELAB are necessarily *not* photometrically 
correct gradients.

* Multiplying colors in CIELAB can't be used to transform the colors "as 
if" a physical color filter had been applied to the scene light source 
when the image was photographed.

* CIELAB channel data is radically different from UserRGB channel data. 
Converting from one RGB color space to the next rearranges channel data, 
"only" introducing crosstalk. Converting from RGB data to CIELAB goes 
way beyond introducing crosstalk and instead re-encodes the data in a 
completely different color space model that intentionally separates 
"opponent colors" from lightness.

* Many useful things that can be done using LAB Levels and Curves 
adjustments. None of them are substitutes for linear gamma RGB Levels 
and Curves adjustments.


7. Moving past unbounded sRGB to HDR scene-referred editing

Currently GIMP is a display-referred image editor. The point of 
display-referred image editing is to be able to "display" an image's 
entire dynamic range on a "device".

During display-referred image editing the "display device" is the user's 
monitor screen. When editing is finished, the "display device" might be 
a paper print, LCD or other display screen, or etc.

When editing an image in a display-referred image editor, the RGB 
working space device "white" (R=G=G=1.0) is mapped to "device white". 
Likewise RGB black (R=G=B=0) is mapped to "device black".

This means the display-referred dynamic range is limited to roughly 9 
stops, depending on where you draw the cut-off between "just noticeably 
dark gray" and "visually the same as black". See "Models for image 
editing: Display-referred and scene-referred" 
(http://ninedegreesbelow.com/photography/display-referred-scene-referred.html).

Real world scenes can have dynamic ranges that exceed 20 stops. In an 
HDR scene-referred image editor, R=G=B=1.0 is just another point on a 
grayscale that runs from 0 (no light) to some very high value that is 
mostly determined by considerations of precision.

If I understand Pippin correctly, he has anticipated the ICC's move to 
eventually accomodate HDR scene-referred image editing and I'm pretty 
sure he's been laying the groundwork for transforming GIMP into an HDR 
"scene referred" image editor.

For GIMP to be an HDR scene-referred image editor, clipping code really 
must be removed and ALL editing must be done using UserRGB channel 
values. Otherwise the user's task of interpreting the resulting RGB 
channel values becomes impossible because the user won't have any way of 
knowing whether RGB channel values are in or out of gamut with respect 
to the UserRGB xy chromaticities.

For GIMP to be an HDR scene-referred image editor and *also* a 
display-referred image editor, of course the user must be able to flip 
between a mode where all UserRGB channel values are clipped to the range 
0.0 to 1.0, and a mode where the channel values are unclipped to 
acccomodate HDR channel values.

Anyway, if I'm correct, and HDR scene-referred image editing really is 
what Pippin has in mind for GIMP, it would be nice to move past 
discussing unbounded *s*RGB (it really is not a good idea) and move on 
to discussing the options that unbounded *User*RGB image editing can 
open up.

In unbounded *User*RGB, a color's RGB channel values and hence Luminance 
(Y value) can be as high as required to capture the scene dynamic range. 
But RGB channel values will never be negative except if the *user* makes 
an editing move that deliberately produces such values for whatever 
purpose the user has in mind.

I've already been making use of some of what high bit depth GIMP would 
be able to do in this respect (for example using the unbounded *User*RGB 
Levels upper sliders), and it opens up some pretty cool editing options 
even for what would otherwise be normal display-referred image editing.

Elle