Re: strange rounding of an arithmetic serie

[Frédéric Parrenin <parrenin ujf-grenoble fr>]

OK I see.

But why this problem does not show up in Calc or XL?

It seems they manage to find a workaround.

2014-10-30 18:13 GMT+01:00 Allin Cottrell <cottrell wfu edu>:

On Thu, 30 Oct 2014, Frédéric Parrenin wrote:

If I type 0 in A1 and 0.1 in A2, select A1:A2 and then drag the bottom
right corner, I get:
0
0.1
0.2
0.3
0.4
0.5
0.6000000000000001

I am using 1.12.14 on a linux 64 bit machine.

Is this a known issue?
If yes, what is the corresponding bug report?

Yes, it's a very well known issue, but there's no corresponding bug report -- because the issue is the inherently finite precision of computer arithmetic. The "double" datatype comprises 64 bits and cannot model the real line. You've just discovered that 0.6000... (with an arbitrary number of trailing zeros) is not a number that can be exactly represented by a double.

Allin Cottrell