Re: Re: Re: Re: deco-Math project, step 00_a: exact bin and dec 'ranges' (in gnumeric).



On Wed, Jul 07, 2021 at 06:16:46PM -0400, Morten Welinder wrote:
Gnumeric's rounding here is indeed done with

    round_to_int(x * 10^d) / 10^d

except that round_to_int deliberately misrounds 0.5-1ulp to 1.  That's
not important here.

Thank you Morten for introducing some reality into this discussion. It's 
good to have confirmation that this is what Gnumeric is doing.



-- 
Steve


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]