On Tue, Nov 1, 2011 at 4:41 PM, Simon Fraser <[email protected]> wrote:
> On Nov 1, 2011, at 4:33 PM, Ojan Vafai wrote: > > As it is, we only run reftests if you run with --pixel. Is anyone opposed > to running them even if you don't pass --pixel. It's a bit weird because > they do ultimately compare images, but they don't have most of the problems > that cause ports to not run pixel tests by default. > > Notably, it was confusing to me: > https://bugs.webkit.org/show_bug.cgi?id=71158. In that case, we could > also just print an error if you manually run a reftest without --pixel, but > it would be great for the project if all ports ran reftests by default. > > > What does this imply? Does enabling reftests turn on some new set of > tests, does it change the way we run some existing set of tests, or does it > add additional steps to an existing set of tests? What kind of new failures > should we expect? > It causes a new set of tests to run. There's a dozen or so reftests in the tree right now. When they fail, you'll see a pixel diff the same way you would with pixel tests, but we don't compare the rendertrees, so there's never a text diff. In theory, none of the current reftests have any platform specific bits, so they should pass for all ports. Currently, the thing that determines whether something is a reftest is whether there is a -expected.html file. So, foo/bar/myreftest.html will have a foo/bar/myreftest-expected.html that should render the same as myreftest.html pixel-wise. As we import reftest suites, we may add other ways of making things reftests (e.g. putting them in a reftests folder?). See https://bugs.webkit.org/show_bug.cgi?id=66295.
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
