*From*: Steven D'Aprano <steve pearwood info>*To*: gnumeric-list gnome org*Subject*: Re: Two questions about the Gnumeric tests:*Date*: Thu, 3 Feb 2022 07:53:24 +1100

On Wed, Feb 02, 2022 at 07:00:45PM +0100, newbie nullzwei via gnumeric-list wrote:

1. could someone please check if it is correct that statfuns.xls in cell F22 calculates 15.075000000000001 as average of 33.9; 23.3; -1.3 and 4.4?

I don't have access to the spreadsheet just now, but the arithmetic mean of those four values, as calculated by Julia and Python: julia> Statistics.mean([33.9, 23.3, -1.3, 4.4]) 15.075000000000001 A naive calculation in Python:

sum([33.9, 23.3, -1.3, 4.4])/4

15.075000000000001 Using high-precision arithmetic on floating point values:

statistics.fmean([33.9, 23.3, -1.3, 4.4])

15.075 And using exact arithmetic (convert the floats to rational numbers; use exact rational arithmetic for the calculation; convert back to float):

statistics.mean([33.9, 23.3, -1.3, 4.4])

15.075 So 15.075 is the exact result, but plus 1 ULP due to rounding errors is acceptable. If your school maths calculates a different value, perhaps you went to a different school than me, or you made a mistake :-) My school maths calculation gives 60.3 as the sum, and dividing by four gives 15.075. -- Steve

**References**:**Two questions about the Gnumeric tests:***From:*newbie nullzwei

[Date Prev][Date Next] [Thread Prev][Thread Next] [Thread Index] [Date Index] [Author Index]