complex : needless loss of precision [mostly fixed]
- From: John Denker <jsd av8n com>
- To: Gnumeric Spreadsheet List <gnumeric-list gnome org>
- Subject: complex : needless loss of precision [mostly fixed]
- Date: Mon, 08 Apr 2013 11:32:59 -0700
Hi Folks --
I have written some code to deal with the following issue:
It turns out that for some floats, converting to a complex
number and then back to a float loses precision. The CNS
(complex number string) representation doesn't use enough
digits. If this is not obvious, a demonstration is here:
http://www.av8n.com/computer/complex-precision.gnumeric
As is my custom, I include two sheets: One with live calculations,
and one with the results pasted by value, to make sure everybody
can see what I'm seeing, even if there are version-to-version
variations.
Using one more digit makes the problem go away.
Here's the basic conceptual issue:
+/* Note there are two different questions:
+ A double can safely represent how many digits? GNM_DIG
+ How many digits can safely represent a double? GNM_DIG_SAFE
+ */
+#define GNM_DIG_SAFE (1+GNM_DIG)
Right now the code to deal with this is bundled in with the
imfourier patch, but I suppose it could be split out if anybody
wanted. Look for references to GNM_DIG_SAFE here:
http://www.av8n.com/computer/imfourier.patch
This is a patch against the current 1.12.1 tarball.
This counts as "mostly" fixed because I still need to track
down everybody who calls the old code and see whether they
need to be calling the new code.
=============
See also next message.
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]