[Date Prev][Date Next] [Thread Prev][Thread Next]
[Thread Index]
[Date Index]
[Author Index]
Re: ver 1.9.4 use more decimal digits
- From: Morten Welinder <mortenw gnome org>
- To: Morten Welinder <mortenw gnome org>, gnumeric-list <gnumeric-list gnome org>
- Subject: Re: ver 1.9.4 use more decimal digits
- Date: Sun, 1 Mar 2009 21:55:16 -0500
2009/3/1 bill lam <cbill lam gmail com>:
> On Sun, 01 Mar 2009, Morten Welinder wrote:
>> > After upgraded to 1.9.4. Diff the gnumeric file with the previous
>> > 1.8.3
>>
>> This is deliberate. We need to store enough decimals to ensure
>> that the number we had will come back unchanged. The number
>
> This is incorrect. The number pairs have the same bit pattern using
> ieee 754 standard so that they are equal.
No, it is correct, but you skipped this: "The number of decimals needed
depends on what number we're trying to write, so we're just using the
minimal number that fits all possible numbers."
> From what I can see in your message, previous versions did not
> deliberately find the minimum digit representation of floating point
> number within the limit of ieee 754 standard.
Correct. We need 17 digits for "double":
welinder toshiba:~/gnome/gnumeric> perl -e 'printf "%.17g\n", 1+2**-52;'
1.0000000000000002
welinder toshiba:~/gnome/gnumeric> perl -e 'printf "%.16g\n", 1+2**-52;'
1
But, yes, for 5925.57 it would have been enough to use 16.
We used to use 15 (I think), so we actually used to drop the occasional
bit. Not Good[tm]. Somewhat embarrassing, in fact.
Morten
[Date Prev][Date Next] [Thread Prev][Thread Next]
[Thread Index]
[Date Index]
[Author Index]