On Aug 18, 2012, at 1:08 AM, Filip Pizlo <[email protected]> wrote:
> I like your idea of having both the result-we-currently-expect and the
> result-we-think-may-be-more-correct to be checked in. I still prefer Dirk's
> naming scheme though.
I think if we had both checked in, the result-we-think-may-be-more-correct
should be named something other than -expected, since it is not, in fact,
expected. That was the basis of my naming scheme.
I think I would be happy with any scheme that had both checked in, and matched
the criteria that you never have a file named -expected that is unexpected. For
example, there could be schemes with no file named expected. If you let it be
verbose, you could have:
Single result:
foo-expected.txt
Possibly-worse current result, possibly-better older result:
foo-expected-failure.txt
foo-unexpected-pass.txt
>
> I get the notion that "expected" always means literally what it seems to mean
> from the standpoint of whether the tooling is silent for the test (actual ==
> expected) or has something to say.
>
> But I think that if the tooling is behaving right, your concern that "a test
> would fail if it did *not* match the "failing" result" would be addressed:
> the tooling could be silent for actual == failing (if a failing file exists)
> but notify you of an "unexpected pass" if actual == expected.
But if you match neither, you get a failure for not matching the "failing"
result. That still strikes me as a little goofy. Not failing is failing, and
getting the expected result is unexpected. I think my extra-verbose naming
scheme above would better match what you suggest the tool UI would do. Maybe
there is a more concise way to get the same point across.
Regards,
Maciej
_______________________________________________
webkit-dev mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo/webkit-dev
