Your question numbered 1 should be directed to LibreOffice.With respect to your
question numbered 2, Morten pointed out that our calculations are on base 2,
but you are still looking at rounding in base 10: if you had only 2 digits
behind the point, you would have0.00 i.e 0.00 in decimal0.01 i.e 0.25 in
decimal0.10 i.e 0.50 in decimal0.11 I.e 0.75 in decimal1.00 i.e 1.00 in
decimalSo where would 0.0111 round to? It is closer to 0.50 than to
0.25.AndreasAndreasSent from my Galaxy
-------- Original message --------From: "b. via gnumeric-list"
<[email protected]> Date: 2021-05-18 22:29 (GMT-07:00) To:
[email protected] Subject: follow up - newbie questions, sum over range,
rounding of special problematic values
hi @all, i'm new here, pls. excuse if i am not well adapted to the manners,
(asked something some time ago, got answer from a 'private email', but none to
follow up questions, thus try to write to the list again, sorry if wrong,
>> 1. gnumeric produces much better sums over ranges than other spreadsheets
>> which i have seen (LO calc and MS excel), can somebody explain by which
>> means this is
>> achieved ('kahan summation' or 'summand ordering' or 'using FPU registers
>> and long doubles' or other?) and / or give me a code pointer?
> 1. We use an extension of Kahan summation, see
https://gitlab.gnome.org/GNOME/goffice/-/blob/master/goffice/math/go-accumulator.c
thank you, i'm not a coder, looks nice and short and - as i have seen - works
very good!,
is it ok to suggest the people at LibreOffice to have a look at this, there is
one who wants to implement Kahan (with moderate success so far), and another is
a little paranoid and thinks LibreOffice would get into trouble just by looking
at your code / that of gnumeric at all, see
https://bugs.documentfoundation.org/show_bug.cgi?id=137679#c42
>> 3. gnumeric produces quite accurate results (3.000000000000000444E-01 for
>> '=0.1 + 0.2'), but has weaknesses when rounding that to e.g. 16 decimals
>> digits, results in
>> 3.000000000000000999E-01 which is 'enhancing the fail'. ...
> 3. This is based on misunderstandings. Our calculations are based on
base 2.
there are other questionable rounding results, e.g.
'=rounddown(0.249999999999999972244,16)' results in 0.250000000000000000000,
while '=roundup(0.249999999999999972244,16)' produces 0.249999999999999888978
which - acc. my simple math knowledge - should be the other way around?
(0.249999999999999972244 results from calculating 0.25 minus one ULP, i'd
expect a 'roundDOWN' always to be less or equal to a 'roundUP', i suspect a
systematic error, but might be wrong in my expectations as well, bear with me
if the latter ... ),
reg.
b.
_______________________________________________
gnumeric-list mailing list
[email protected]
https://mail.gnome.org/mailman/listinfo/gnumeric-list
