# Re: Two questions about the Gnumeric tests:

• From: Steven D'Aprano <steve pearwood info>
• To: gnumeric-list gnome org
• Subject: Re: Two questions about the Gnumeric tests:
• Date: Thu, 3 Feb 2022 07:53:24 +1100

```On Wed, Feb 02, 2022 at 07:00:45PM +0100, newbie nullzwei via gnumeric-list wrote:

```
```   1. could someone please check if it is correct that statfuns.xls in cell
F22 calculates
15.075000000000001 as average of 33.9; 23.3; -1.3 and 4.4?
```
```
I don't have access to the spreadsheet just now, but the arithmetic mean
of those four values, as calculated by Julia and Python:

julia> Statistics.mean([33.9, 23.3, -1.3, 4.4])
15.075000000000001

A naive calculation in Python:

```
```sum([33.9, 23.3, -1.3, 4.4])/4
```
```15.075000000000001

Using high-precision arithmetic on floating point values:

```
```statistics.fmean([33.9, 23.3, -1.3, 4.4])
```
```15.075

And using exact arithmetic (convert the floats to rational numbers; use
exact rational arithmetic for the calculation; convert back to float):

```
```statistics.mean([33.9, 23.3, -1.3, 4.4])
```
```15.075

So 15.075 is the exact result, but plus 1 ULP due to rounding errors is
acceptable. If your school maths calculates a different value, perhaps
you went to a different school than me, or you made a mistake :-)

My school maths calculation gives 60.3 as the sum, and dividing by four
gives 15.075.

--
Steve
```

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]