Re: ver 1.9.4 use more decimal digits



2009/3/1 bill lam <cbill lam gmail com>:
On Sun, 01 Mar 2009, Morten Welinder wrote:
After upgraded to 1.9.4. Diff the gnumeric file with the previous
1.8.3

This is deliberate.  We need to store enough decimals to ensure
that the number we had will come back unchanged.  The number

This is incorrect. The number pairs have the same bit pattern using
ieee 754 standard so that they are equal.

No, it is correct, but  you skipped this: "The number of decimals needed
depends on what number we're trying to write, so we're just using the
minimal number that fits all possible numbers."

From what I can see in your message, previous versions did not
deliberately find the minimum digit representation of floating point
number within the limit of ieee 754 standard.

Correct.  We need 17 digits for "double":

welinder toshiba:~/gnome/gnumeric> perl -e 'printf "%.17g\n", 1+2**-52;'
1.0000000000000002
welinder toshiba:~/gnome/gnumeric> perl -e 'printf "%.16g\n", 1+2**-52;'
1

But, yes, for 5925.57 it would have been enough to use 16.

We used to use 15 (I think), so we actually used to drop the occasional
bit.  Not Good[tm].  Somewhat embarrassing, in fact.

Morten



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]