Re: [Gimp-developer] GIMP's future internal image data format

[Bogdan Szczurek <thebodzio gmail com>]

> I don't want to hijack Alexandre's thread with the interesting
> discussion that started therein, so here's a new thread for it.

I think you did a good thing by creating this thread.

> 2012/1/27 Bogdan Szczurek<thebodzio gmail com>:
>> W dniu 12-01-27 10:16, Alexandre Prokoudine pisze:
>>> On Fri, Jan 27, 2012 at 11:01 AM, Martin Nordholts wrote:
>>>> Images shall always be composed in 32-bit floating point
>>>> RGBA
>>
>> Which RGB? Is it scRGB of GEGL "guts"? :)
>
> Hi thebodzio
>
> In the end, yes, GIMP's native image data format will likely be 32 bit
> floating point per component/channel, pre-multiplied linear light RGBA
> with the same primaries and white point as scRGB (and consequently
> sRGB).

I thought so, but wasn't sure. Anyway… I *wanted* to be sure since 
graphics editing without solid colorimetric interpretation is just a 
child's play :).

>>>> and then have suitable filters and export mechanisms to deal with
>>>> grayscale and indexed images.
>>
>> …and 16 bits per channel… and 8 bits per channel… and bitmaps (1 
bit)… and
>> multichannel… and CMYKs (I wrote some time ago about them on this list)…
>>
>> I see two problems with that.
>>
>> First is memory use. I'll give an example from my own backyard about
>> bitmaps. Contrary to what may appear bitmaps are still very 
important image
>> mode. At my work I use them as both small and precise means for 
preparing
>> and storing scanned pages of old books destined to be reprinted. 
Given I'll
>> be compelled to use 32 bits for each pixel I'll practically "waste" 
31 bits.
>> So my images (of course at the time of manipulating them in GIMP) 
will take
>> 32 times more space than necessary. It may seem harmless, but take into
>> account that such bitmaps are oftentimes of sizes about 8000 x 10000 
pixels.
>> Pretty much the same crude calculations are true for other "pixel 
depths" as
>> well (8 bits – 4 times, and so on).
>
> First, I must point out that your arguments needs to be derived from
> the GIMP product vision [1], not personal requirements. Saying "I need
> GIMP to do X" is not a valid argument for adding support for X in
> GIMP, you need to say "GIMP needs to do X to fulfill the product
> vision".

What I'm trying hard to say is: "I need GIMP to do X, *because*…" :). 
You're right – my opinion is personal and it is inteded as such. I speak 
out of my experience with graphics and design for printing business 
(tiny company as it is but still :)). I see it as a discussion between 
craftsman and tool supplier. I need "a hammer shaped this way". Tool 
supplier answers "okay, but I think it would be better to adjust the 
shape". So discussion follows :).

At the same time, I can't emphasize enough that I'm not, by no means, 
trying to usurp the position of "absolute truth" holder. What is more: 
I'm strongly against such stance, so if I'm proven wrong or I can't 
convince others… so be it – maybe in time I'll be able to provide 
stronger arguments. Anyway – it would be wrong of me to not to try to 
help in development of app I see as one of the most important for the 
whole OSS movement. Also selfish to not share my experiences and findings.

Now… excuse me if I sound a bit heretic or provocative (I simply don't 
know how to ask that question in other way), but… what exactly is GIMP's 
vision? I mean, I've seen roadmap, features, short info on the gimp.org 
and developer wiki, yet I couldn't find any piece of text that could 
pass as a *paramount* of GIMP's development. Granted I didn't do 
thourough investigation only a quick look into the matters, but still. 
Maybe there is something I missed, in that case I'm sorry to bring that 
up, but maybe GIMP's vision exists only as a nonwritten idea – clear to 
devs and frequent followers of this list, but not so for the rest of the 
world. If the latter is true, then maybe verbalizing "GIMP's vision" 
would be a good thing to do? Even if only to put an end to a questions 
like mine.

> Otherwise GIMP is doomed to begin (keep?) delivering an
> inconsistent user experience in some areas.

I agree that "fixed point" is needed, but as I undestand it becomes 
"fixed" as a result of a discussion – hence my post. My arguments may be 
found insufficient or inconvincing – I'm ready to face that – but I'm 
sure that at least they'll provide some point of view and provoke a bit 
of "thinking" :).

> There are two solutions to the memory usage problem:
>
> 1. Buy more RAM. GIMP does not need to run well on a 512 MB RAM
> machine because it is reasonable to expect users of a high-end photo
> manipulation program to acquire sufficient amounts of RAM. NOTE: I'm
> not saying GIMP should be wasteful with memory...

I agree completely! I know RAM is easily accessible for a reasonable 
price, but, as you've said – it's no reason to be wasteful with it. 
Especially the latter is my concern.

I want to make a small "reasoning". If each image will be stored (stored 
all the time – not only "converted" to 32 bits fp during manipulation) 
as 32 bits fp RGBA (specific colorimetric being non-issue here), then 
each pixel ("light" sample) will be represented as qudruplet of 32 bit 
values, so each pixel will require 16 bytes (I assume octet bytes of 
course). Now I want to consider 24 MP image of size e.g. 6000 x 4000 px. 
I think such size is reasonable to assume, since full frame DSLRs at 
tops deliver images of similar size. Simple calculation gives: 6000 x 
4000 x 16 = 384 000 000 bytes (about 366 MiB). It's just "raw" image. 
But now… add to that a couple of layers ("static" or "indestructive"), 
some undo levels… Quite recently I had some scanned map to process 
(RGB). It wasn't the biggest material I had in my hands, even so it was 
about 18000 x 14000. It was a "flat" scan – no alpha, but since we 
intend to add alpha anyway we can again count it in. If we'd use 32 bits 
fp per channel we'd get 3.76 GiB (IMCAC) for starters – just to hold 
image in RAM. With such sizes RAM is still an issue.

> 2. Make GIMP clever. If GIMP encounters a tile with only values 0.0
> and 1.0, the 32 bpc data can be transparently, i.e. without the user
> noticing, replaced with 1 bpc data. As soon as more bits of precision
> is required to avoid loss of data, GIMP can transparently convert the
> tile back to RGBA float. The same kind of optimization can be done for
> completely black, white and transparent tiles too.

This is promising but image mode is not just about keeping image istelf 
as small as possible. It's also to make sure I won't use any colors I 
don't want to use. E.g. when I'm editing grayscale, I don't want to by 
mistake use, say, red. "Smart" mechanism would just "convert" image so 
it would be able to hold my red and wouldn't even warn me about it. Why 
bother? It's not so much of a problem with "screen" graphics, but in 
print you know exactly that you can use only one colorant for this or 
that image (presumably process black) and that's it. If you won't keep 
your palette under control you're asking for a problem.

The same story is with bitmap images. If I'm correcting scanned text I 
wan't to keep my colors constrained to white and black only. 
Antialiasing done with more colors looks nice on screen, but in print 
it'll cause RIP to turn some pixels into halftones, and… there goes your 
nice, sharp edges – text will seem a kind of "blurred". If I'd, by 
mistake, used "soft" brush I'd be exactly in that kind of situation. If 
colors aren't keep in order by image mode (or palette constraints) I 
won't be able to notice some "gray" pixels and "smart conversion" will 
put image into grayscale. Sure, I can use some additional filters before 
saving or even while saving/exporting, but in former I'd have to remeber 
to do it and in latter it'll complicate save procedure.

And finally… it can be processing intensive task.

> Maintaining support in GIMP for an internal image data format of 8 bpc
> or adding support for an native image data format of 16 bpc is silly
> because such formats is going to result in rounding errors and lack of
> HDR support, which is not high-end.

HDR or not it'll most of the time end up on (again: most of the time :)) 
8 bit driven display. Besides OpenEXR use exactly 16 bpc (mantissa 10 
bits to be exact). HDR power is not in images that inherently look 
better but in greater capability of processing them (e.g. about +- 30 EV 
in OpenEXR!).

>> I understand "filters" and "export mechanisms" are to be basically 
means of
>> "cramming" information from fp pixels into "less precise" units. It 
means
>> it's very probable some information will be lost/distorted. It will 
happen
>> in more or less dramatic way, but the problem remains: what tools do 
we need
>> to give to user to enable them *complete* and *precise* control over
>> "conversion" results (meaning: this pixel should have *exactly* this 
or that
>> value). When you stick to "image modes" as your paradigm in app 
itself, you
>> don't have to worry much about it – user "gets what he sees" 
(granted he's
>> into proper CM workflow ;)) or at least have absolute control over 
sample
>> values right "on the canvas". Moving this control from "canvas" to 
"export"
>> will result in another layer of complexity with (probably) reduced
>> capabilities for the sake of "simplicity". Most of the time it'll 
probably
>> work (much like CM), but I think it'll be a hell to debug and enemy of
>> marginal, yet important use cases.
>
> If you want a pixel to be RGB u8 (128, 90 90) when you export to a
> PNG8, simply paint that pixel RGB u8 (128, 90 90). There are no
> problems for RGBA float to represent RGB u8. Maybe I don't fully
> understand what you mean, could you give a concrete and clear use case
> that illustrates a problem?

That's the problem too. Let's assume we have RGBA as internal sample 
format. Each channel is 32 bits fp. It means value of each channel is 
represented by (IEEE 745): 1 bit of sign, 8 bits of exponent, 23 bits of 
really significant value. Now, how do we interpret this value 
colorimetrically?

We can say: 0 – dark, 255 – bright (assuming additive color model and 
leaving some less significant details aside). It would mean that between 
0 and 1 we use 1 bit of our "value space" for integer, leaving 22 bits 
for mantissa. Between 255 and 254 however, we have to sacrifice 7 bits 
of significand for integer value and 16 bits for mantissa. It's 16 vs. 
22 bits of precision between values. 16 bits is still plenty of values, 
but still less than 22. So "subvalues" would be unevenly distributed. 
Will it affect resulting image and when? I don't know, but it's good to 
take this into account and test it.

We can also state: 0 for dark, 1 for bright (maximum). In this case, 
we'd have evenly distributed "subvalues" (beside 0, 22 bits of 
significand), but then again we'd have most of exponent bits practically 
unused. These bits could be otherwise put in use to make value even more 
precise. Having said all that I wonder if a) such precision isn't an 
overkill, b) wouldn't we be better off with integers.

Even if we'd disregard all else. Let's assume we use RGBA, 32 bits fp 
per channel and disregard "color mode" at all. What about some really 
useful color models, like Lab, XYZ, HSV, HSL, CMYK and also 
"multichannel"? Especially in two latter, there are situations when 
you'd need to set "this channel" to "that value exactly". If all data is 
RGB, how we'd provide this possibility? We'd have to temporarily derive 
channel value from RGB, modify it, and convert back to RGB. And all that 
assuming that all operations are bijective between RGB and other color 
space which in practice simply *isn't true* most of the time.

I really want to see at least CMYK and multichannel in GIMP. Without 
them we can hardly think about using GIMP in serious (though rare) 
applications in publishing workflows. I won't say of it more (if not 
requested) – it was discussed before, maybe it's just not GIMP 
development objective :).

> / Martin

Great to have constructive discussion with you!

My best!
thebodzio