> On 18 Dec 2021, at 17:51, Steve Kargl <s...@troutmask.apl.washington.edu> > wrote: > > On Sat, Dec 18, 2021 at 10:41:14AM +0000, Mark Murray wrote: >> >> Hmm. I think my understanding of ULP is missing something? >> >> I thought that ULP could not be greater than the mantissa size >> in bits? >> >> I.e., I thought it represents average rounding error (compared with >> "perfect rounding"), not truncation error, as the above very large >> ULPs suggest. >> > > The definition of ULP differs according which expert you > choose to follow. :-) For me (a non-expert), ULP is measured > in the system of the "accurate answer", which is assumed to > have many more bits of precision than the "approximate answer". > From a very old das@ email and for long double I have
<snip> Thank you! I checked the definition that I was used to, and it is roughly "how many bits of the mantissa are inaccurate (because of rounding error)". I can see how both work. For utterly massive numbers like from Gamma(), I can see how accounting for a much larger range works. It still feels slightly tricky, as e.g. how many digits after the floating point do you account for? > I don't print out the hex representation in ld128, but you see > the number of correct decimal digits is 33 digits compared to > 36. Looking good! M -- Mark R V Murray
signature.asc
Description: Message signed with OpenPGP
