Re: Two questions about the Gnumeric tests:

[Steven D'Aprano <steve pearwood info>]

On Wed, Feb 02, 2022 at 07:00:45PM +0100, newbie nullzwei via gnumeric-list wrote:

>    1. could someone please check if it is correct that statfuns.xls in cell
>    F22 calculates
>    15.075000000000001 as average of 33.9; 23.3; -1.3 and 4.4? 

I don't have access to the spreadsheet just now, but the arithmetic mean 
of those four values, as calculated by Julia and Python:

julia> Statistics.mean([33.9, 23.3, -1.3, 4.4])
15.075000000000001

A naive calculation in Python:

> > > sum([33.9, 23.3, -1.3, 4.4])/4
15.075000000000001

Using high-precision arithmetic on floating point values:

> > > statistics.fmean([33.9, 23.3, -1.3, 4.4])
15.075

And using exact arithmetic (convert the floats to rational numbers; use 
exact rational arithmetic for the calculation; convert back to float):

> > > statistics.mean([33.9, 23.3, -1.3, 4.4])
15.075

So 15.075 is the exact result, but plus 1 ULP due to rounding errors is 
acceptable. If your school maths calculates a different value, perhaps 
you went to a different school than me, or you made a mistake :-)

My school maths calculation gives 60.3 as the sum, and dividing by four 
gives 15.075.


-- 
Steve