On 4/7/2014 9:02 AM, Aryeh Gregor wrote: > On Mon, Apr 7, 2014 at 3:20 PM, Andrew Halberstadt > <ahalberst...@mozilla.com> wrote: >> I would guess the former is true in most cases. But at least there we have a >> *chance* at tracking down and fixing the failure, even if it takes awhile >> before it becomes annoying enough to prioritize. If we made it so >> intermittents never annoyed anyone, there would be even less motivation to >> fix them. Yes in theory we would still have a list of top failing >> intermittents. In practice that list will be ignored. > Is this better or worse than the status quo? Just because a bug > happens to have made its way into our test suite doesn't mean it > should be high priority. If the bug isn't causing known problems for > users, it makes sense to ignore it in favor of working on bugs that > are known to affect users. Why not let the relevant developers make > that prioritization decision, and ignore the bug forever if they don't > think it's as important as other things they're working on? > > If a bug is causing a test to fail intermittently, then that test loses value. It still has some value in that it can catch regressions that cause it to fail permanently, but we would not be able to catch a regression that causes it to fail intermittently.
It's difficult to say whether bugs we find via tests are more or less important than bugs we find via users. It's entirely possible that lots of the bugs that cause intermittent test failures cause intermittent weird behavior for our users, we simply don't have any visibility into that. -Ted _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
