- From: "Andreas J. Guelzow" <aguelzow math concordia ab ca>
- To: gnumeric-list gnome org
- Subject: Re: Rounding??
- Date: Tue, 12 Jun 2001 17:29:17 -0700
Morten Welinder wrote:
> Gnumeric seems to do all sorts of things I cant quite figure out.
> The _important_ thing is to have _consistent_ behaviour.
> At the moment
> 3.15, 3.25, 3.35, 3.45, 3.55, 3.65, 3.75, 3.85, 3.95, 4.05, 4.15
> gives me
> 3.1, 3.2, 3.3, 3.5, 3.5, 3.6, 3.7, 3.8, 4.0, 4.0, 4.2
> which just seems confusing. Why are 3.4 and 3.9 missed??
> Think binary.
That's what I suspected (without doing the arithmetic), but the internal
representation should _not_ influence the expected rounding behaviour.
Without having looked at the source I suspect that the remainder is
simply being compared with 5,...
It should really cut at a value wich causes the expected behaviour for
decimals that are only a few digits longer. (There will probably always
be the problem that some values very close to the cutoff will round
Prof. Dr. Andreas J. Guelzow
Chair of Science, Assoc. Prof. of Math.
Concordia University College of Alberta
Assoc. Prof. of Math.
] [Thread Prev