Re: Pixel precision

2012-10-09 Thread Robert O'Callahan
I guess 7 digits of precision beyond the decimal point is overkill. How about we output values rounded to the nearest 1e-6. Rob -- “You have heard that it was said, ‘Love your neighbor and hate your enemy.’ But I tell you, love your enemies and pray for those who persecute you, that you may be c

Re: Pixel precision

2012-10-08 Thread Anthony Jones
On 09/10/12 17:37, Robert O'Callahan wrote: > Which tests? With 7 digits of float precision it fails: * content/smil/test/test_smilCSSFromBy.xhtml * layout/style/test/test_animations.html * layout/style/test/test_bug399349.html * layout/style/test/test_flexbox_layout.html * layout/style/test/test

Re: Pixel precision

2012-10-08 Thread Robert O'Callahan
On Tue, Oct 9, 2012 at 10:16 AM, Anthony Jones wrote: > This formats the float to 6 significant figures, however a float has 7.2 > significant figures[1]. A float can contain any integer up to 2^24. > Anything more than 999,999 pixels shows in exponent format as 1e+6 and so > on. Currently co-ord