Re: [Gimp-developer] How precise is Gimp 32-bit floating point compared to 16-bit integer?

On Monday, 16. December 2013 11:59:20 Elle Stone wrote:
To restate the question, in decimal notation, 1 divided by 65535 is
0.00001525878906250000. So 16-bit integer precision requires 16 decimal
places (lop off the four trailing zeros) in floating point to express
the floating point equivalent of 1 16-bit integer tonal step, yes? no?

Not quite:

0/(2^16-1) = 0
1/(2^16-1) ≈ 0.000015259022
2/(2^16-1) ≈ 0.000030518044
3/(2^16-1) ≈ 0.000045777066
(2^16-1)/(2^16-1) = 1

So it is quite sufficient to show the first 6 digits after the decimal point, 
any error after that is smaller than 2%, and even that only in the worst case 
of the first step.

Actually the big advantage of floating point numbers (IMHO) is that they do 
not have a linear precision, but each exponentially in-/decreasing times-2-
interval (0, 1/2^8, 1/2^7, ..., 1/2, 1) is split into 2^23 linear steps.  

Since our senses work basically logarithmically, this means that we perceive 
the steps to be of about equal size -- except that we cannot distiguish such 
small steps visually anymore anyways, be it float32 or int16, as pointed out 
by Simon already.


Mein öffentlicher Schlüssel / My public key: 4096R/600ACB3B 2012-04-01
Fingerabdruck / Fingerprint:
9902 575B B9A0 C339 CFDF  250B 9267 CA6B 600A CB3B
Runterladen z.B. bei/ Get it e.g. from:,,,, ...

Attachment: signature.asc
Description: This is a digitally signed message part.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]