On Tue, Nov 13, 2012 at 12:16 PM, Tony Chang <[email protected]> wrote: > On Tue, Nov 13, 2012 at 12:04 PM, Darin Adler <[email protected]> wrote: >> >> On Nov 13, 2012, at 12:02 PM, Tony Chang <[email protected]> wrote: >> >> > I don't think we should support port specific ref test results. That >> > kind of misses the point of using a ref test in the first place. I mean, >> > you >> > may as well check in port specific pixel results which are easier to review >> > for correctness. >> >> I don’t agree that pixel results are easier to review for correctness. > > > Here is a ref test result from ietestcenter: > http://trac.webkit.org/browser/trunk/LayoutTests/ietestcenter/css3/flexbox/flexbox-flex-002-expected.htm > > Looking at that HTML file, it's not immediately obvious that the result is > correct. If I had a png file, it would easy to see if there's red showing > or not. >
Actually reviewing the rendered pixels makes some things easier, but it can be difficult to figure out which rendered pixels matter and which don't. Even in the case of "if you see any red", it can be easy to miss one red pixel on an edge (and of course, there's issues around red-green colorblindness). Plus, you have to consider both the cost of reviewing the initial "correct output" and then the cost of reviewing changes to that output. in theory, we'll get far fewer changes to the correct output that need to be reviewed. For example, if we change how text is rendered (cough *mountain lion* cough) we'll need a new PNG but no change to the ref test. So, that's one less review you have to do. That said, I don't think there's a blanket rule here; I suspect some things are easier to review as pixel tests and some aren't. -- Dirk _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo/webkit-dev
