On Sun, Nov 11, 2012 at 4:21 PM, Neil <n...@parkwaycc.co.uk> wrote: > Why not simply verify that the list of actual fails equals the list of > expected fails, and report items that are only in one of the two lists?
That would be a bit more robust, yes, and it should be doable without much work. It still wouldn't detect the case where different sets of passing tests are run, such as because a change accidentally disabled a large chunk of tests. The system James seems to be describing would detect that condition too. On the other hand, all our existing mochitests don't detect it and we seem to do fine. (There *are* cases where someone accidentally disabled a bunch of tests and no one noticed. Some months ago I found a number of those when I changed mochitests to fail if the file ran no tests. But that only detects if all tests in the file are disabled, not if only a handful are disabled. Still, it's not a common enough scenario that it's worth spending much effort on, IMO.) _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
