I've been tracking down an issue that led me to
nsROCSSPrimitiveValue::GetCssText()
case CSS_PX :
{
float val =
nsPresContext::AppUnitsToFloatCSSPixels(mValue.mAppUnits);
tmpStr.AppendFloat(val);
tmpStr.AppendLiteral("px");
break;
}
This formats the CSS pixel width by calling
nsTSubstring_CharT::AppendFloat()
void AppendFloat( float aFloat )
{ DoAppendFloat(aFloat, 6); }
This formats the float to 6 significant figures, however a float has 7.2
significant figures[1]. A float can contain any integer up to 2^24.
Anything more than 999,999 pixels shows in exponent format as 1e+6 and
so on. Currently co-ordinates are limited to nscoord_MAX = 2^30. This
works out at 2^24 * 64 / 60 which is slightly more than a float can
hold. Changing AppendFloat to 7 digits of precision breaks tests that
assume 6 digits of precision.
I have two questions.
1. Why do we throw away the 7th digit of float precision?
2. How much precision should we support for pixel sizes?
[1] http://en.wikipedia.org/wiki/Floating_point#Internal_representation
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
