Re: bug in gnumeric: rounding error



While I understand the explanation I am surprised that even if I ask for
25 decimal digits it doesn't show that it is slightly smaller than 22
but if I take its difference from 22 (the integer) it shows a small
difference in the 7th or 8th digit.

The difference is about 3.55e-14.  That's no far out in terms of significant
digits that Gnumeric decides that if you ask for more decimals you get
zeros instead of random digits.

Morten



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]