On 07/29/2010 08:33 AM, Rainer Tammer wrote: >> ASSERT (result == 10.0); >> >> Weird - I've never seen that line fail before. If you have a debugger, >> can you put a breakpoint on that line and print the actual value of >> result? > I can do that. > > main(), line 234 in "test-strtod.c" > > (dbx) print result > 10.000000000000002
Yep - rounding inaccuracies have crept in, even for something as simple as 1 * pow(10,1). Which gives me hope that latest gnulib will be immune, since we ditched pow(). But it also makes me wonder who's at fault for introducing a rounding error on such a seemingly simple operation - hardware, compiler, or former gnulib algorithm? -- Eric Blake ebl...@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
