https://bugs.documentfoundation.org/show_bug.cgi?id=157856
Mike Kaganski <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.documentfounda | |tion.org/show_bug.cgi?id=10 | |7953 --- Comment #3 from Mike Kaganski <[email protected]> --- (In reply to cppjohn from comment #0) > The “Sum:” value for “=0.95-0.9” on OpenOffice calc is 0.05 instead of > 0.0499999999999999. It is indeed unclear if this occurrence of "OpenOffice" in comment 0 actually meant OpenOffice (better use AOO, to avoid confusion in the future), or was meant to refer to LibreOffice (LO). But from context, it seems to mean AOO, as showing "correct" answer; and so, bug 107953 seems relevant to *this specific bit* related to display. This faq has a general overview: https://wiki.documentfoundation.org/Faq/Calc/Accuracy It indeed not only relates to Calc, but to any floating-point processing. It might happen, that we dropped some rounding, that allowed to compare some different floating-point values equal; this would need checking (so please do attach a sample ODS, with the macros; it is unclear what you did to get the values from the cells - e.g., one could take strings, and convert to numbers; however "clear" the textual description is - it is unreasonable to ask people to re-invent the macros, and *hope* they have the same as you have). This calculator allows to see the binary representations of doubles: http://weitz.de/ieee/. It helps to see, that 0.05 + 0.90 results in 0.9500000000000001 (binary 0b0011111111101110011001100110011001100110011001100110011001100111), while 0.95 is 0b0011111110101001100110011001100110011001100110011001100110011010 (the difference is in the last digit). -- You are receiving this mail because: You are the assignee for the bug.
