Re: compatibility problem between 'double' and 'long double' versions | was: follow up: Aw: Re: 'testing' in gnumeric

[Morten Welinder <mortenw gnome org>]

For .ods there is no way to store what number system is in use.  You
could certainly store the shorter "0.1", but that would simply be
wrong in a different way.

The number you have in hand is
0.1000000000000000055511151231257827021181583404541015625 aka
3602879701896397 / 2^55.  There simply is no way of knowing whether
that was meant to be 0.1 or slightly more.  It looks like we have
chosen to store just enough digits to ensure round-trip for double.

For the .gnumeric format, you can use the stored FloatRadix and
FloatDigits (see xml_write_number_system) to decide what to do.

Again: the problem you are trying to solve does not have a solution.
You are trying to look at 3602879701896397 / 2^55 and make the
decision "that's not the right number; it should have been 1/10
exactly".  But since you don't know where that value came from you
cannot know whether you are reducing or enlarging the error of
representation.

M.