Calculation accuracy
- From: so09 gmx de
- To: Gnumeric Mailing List <gnumeric-list gnome org>
- Subject: Calculation accuracy
- Date: Wed, 29 Feb 2012 08:43:02 +0100
Dear all,
I have a question regarding calculation accuracy (version 1.10.17,
latest Ubuntu).
"=if(condition;A*C;B*C)" and "=if(condition;A;B)*C" bring upon slightly
different results. The results differ only in the last decimal place(s),
which wouldn't make much of a difference at all. I check for "=max()"
value of an array though and in this situation the tiny, tiny difference
in a decimal place(s) does make a difference.
So how does Gnumeric work internally? Does "=if(condition;A;B)*C"
truncate or round values A and B somehow?
Regards
Soeren
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]