On Monday, 16. December 2013 11:59:20 Elle Stone wrote:
To restate the question, in decimal notation, 1 divided by 65535 is 0.00001525878906250000. So 16-bit integer precision requires 16 decimal places (lop off the four trailing zeros) in floating point to express the floating point equivalent of 1 16-bit integer tonal step, yes? no?
Not quite: 0/(2^16-1) = 0 1/(2^16-1) ≈ 0.000015259022 2/(2^16-1) ≈ 0.000030518044 3/(2^16-1) ≈ 0.000045777066 ... (2^16-1)/(2^16-1) = 1 So it is quite sufficient to show the first 6 digits after the decimal point, any error after that is smaller than 2%, and even that only in the worst case of the first step. Actually the big advantage of floating point numbers (IMHO) is that they do not have a linear precision, but each exponentially in-/decreasing times-2- interval (0, 1/2^8, 1/2^7, ..., 1/2, 1) is split into 2^23 linear steps. Since our senses work basically logarithmically, this means that we perceive the steps to be of about equal size -- except that we cannot distiguish such small steps visually anymore anyways, be it float32 or int16, as pointed out by Simon already. Cheers, Daniel -- Mein öffentlicher Schlüssel / My public key: 4096R/600ACB3B 2012-04-01 Fingerabdruck / Fingerprint: 9902 575B B9A0 C339 CFDF 250B 9267 CA6B 600A CB3B Runterladen z.B. bei/ Get it e.g. from: pgp.mit.edu, subkeys.pgp.net, pgp.uni-mainz.de, pool.sks-keyservers.net, ...
Attachment:
signature.asc
Description: This is a digitally signed message part.