On Thu, Aug 16, 2012 at 2:32 PM, Filip Pizlo <[email protected]> wrote:
> 1) Switching to skipping flaky tests wholesale in all ports would be > great, and then we could get rid of the flakiness support. > Then you don't notice when a flaky tests stops being flaky. The cost of flakiness support on the project does not seem large to me and it's pretty frequent that a test will only be flaky for a few hundred runs (e.g. due to a bad checkin that gets fixed), so we can then remove it from TestExpectations and run the test as normal. 2) The WONTFIX mode in TestExpectations feels to me more like a statement > that you're just trying to see if the test doesn't crash. Correct? Either > way, it's confusing. > WONTFIX is for tests that don't make sense to fix for the given port (e.g. dashboard-specific tests for non-Apple ports). It's a way of distinguishing tests that we're skipping because of a bug on our part vs. tests that we're skipping because the test doesn't apply to the port. 3) Your new mechanism feels like it's already covered by our existing use > of -expected files. I'm not quite convinced that having -failing in > addition to -expected files would be all that helpful. > But there are many cases where we *know* the result is incorrect, e.g. it doesn't match a spec. Sure, there are also many cases where it's not clear what the correct behavior is.
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo/webkit-dev
