Re: Trouble getting 30 decimal places

[Jean Brefort <jean brefort free fr>]

It's a precision issue. Gnumeric uses C double to store real numbers and
you can't expect more that 17 decimals. Compiling with
--with-long-double would give a better precision, but this has always
been marked as experimental.

Regards,
Jean

Le lundi 16 juillet 2012 à 19:24 -0400, Clay Lawrence a écrit :
> Can someone else try this and see if they get the same results I do.
> I'm using Gnumeric Spreadsheet 1.10.16
>
>
> Format a cell to 30 decimal places and enter =1/sqrt(2)
> I get 0.707106781186547460000000000000 which rounds after 17 places
> instead of 30.
>
>
> I also went to WolframAlpha and calculated 1/√2 to 30+ places;
> 0.7071067811865475244008443621048490392848359376884740 but after
> keying this in directly Gnumeric still rounds it to
> 0.707106781186547570000000000000 as soon as I hit the enter key. 
>
>
> Is there a setting somewhere that I can set to get Gnumeric to use all
> of the 30 of the places it displays instead of rounding to 17 places
> and padding the remainder with zeroes?
>
>
> -- 
>
> Clay
>
> "There are only 10 kinds of people in the world --
>    Those who understand binary, and those who don't."
>
>
>
> _______________________________________________
> gnumeric-list mailing list
> gnumeric-list gnome org
> https://mail.gnome.org/mailman/listinfo/gnumeric-list