Søren Sandmann <[email protected]> writes: > The correct way to convert from floating point to integers is then to > pick the integer i such that its corresponding value is as close as > possible to the given floating point value. In other words, the > conversion should be done by rounding, and not by dividing the input > range into equal-sized sections.
Hi Søren, Thanks for bringing up this issue. It's fun to take another look at an issue I haven't examined closely in quite some time. (And it's heartening to see that I documented my thoughts fairly carefully at the time.) When considering these conversions from integers to floating-point and back I think we[*] universally agree that all integer values must round trip through both conversions unchanged. What you advocate here, (and what Kalle also proposes), is that the conversion should be chosen in such a way as to also minimize the error when round-tripping a floating-point value to an integer and back. I can't find evidence that I considered this metric in my argument from 2006, but it is a metric that makes sense to minimize. So that much seems sound. There's one part of your position that I'm unclear on. You say: > But the output integers have inherent values, where 0x0000 corresponds > to 0.0, 0xffff corresponds to 1.0 and 0x0001 corresponds to > 1/65535.0. That is not an arbitrary convention -- when these integers > are stored in framebuffers or .PNG files they are (at least in > principle) the assumption is that they correspond to these values. The > reason for choosing the sample points in this way, is that it is > important to be able to represent the particular values 0.0 and 1.0 > exactly. I agree that 0x0000 must correspond to 0.0 and that 0xffff must correspond to 1.0. But then you simply assert that the conversion from integer to floating-point must be f(i) = i/65535.0. What's the justification for this? Other functions could be chosen for the integer->floating-point direction to minimize the round-trip error for floating-point values while also not violating the round-trip equivalence of integer values. For example, Kalle proposes other functions than i/65535.0. Is the argument that this conversion is so simple that it's safe to assume that that's what the user had in mind when storing the integer value into the PNG file? I'm not necessarily arguing against the change here. I'm just pointing out that the explanation for the change seems to depend on an implicit assumption without much justification. Meanwhile, I am curious what brought about your interest in changing this function. What actual problem are you trying to solve here? -Carl [*] Here I'm referring at least to the following which I reviewed: Your message to cairo@ here My message to cairo@ in 2006 Owen's web page which I referenced in my 2006 post The PDF by Kalle Rutanen referenced elsewhere in this thread
pgp9aDN_nLq8k.pgp
Description: PGP signature
_______________________________________________ Pixman mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/pixman
