complex : needless loss of precision [mostly fixed]



Hi Folks --

I have written some code to deal with the following issue:

 It turns out that for some floats, converting to a complex
 number and then back to a float loses precision.  The CNS
 (complex number string) representation doesn't use enough 
 digits.  If this is not obvious, a demonstration is here:
   http://www.av8n.com/computer/complex-precision.gnumeric

 As is my custom, I include two sheets:  One with live calculations,
 and one with the results pasted by value, to make sure everybody
 can see what I'm seeing, even if there are version-to-version
 variations.

 Using one more digit makes the problem go away.

 Here's the basic conceptual issue:

+/* Note there are two different questions:
+   A double can safely represent how many digits?  GNM_DIG
+   How many digits can safely represent a double?  GNM_DIG_SAFE
+ */
+#define GNM_DIG_SAFE (1+GNM_DIG)

 Right now the code to deal with this is bundled in with the 
 imfourier patch, but I suppose it could be split out if anybody 
 wanted.  Look for references to GNM_DIG_SAFE here:
    http://www.av8n.com/computer/imfourier.patch

 This is a patch against the current 1.12.1 tarball.

 This counts as "mostly" fixed because I still need to track
 down everybody who calls the old code and see whether they
 need to be calling the new code.

=============

See also next message.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]