Gregory Szorc writes: > 2) Run marked intermittent tests multiple times. If it works all > 25 times, fail the test run for inconsistent metadata.
We need to consider intermittently failing tests as failed, and we need to only test things that always pass. We can't rely on statistics to tell us about changes in test results without huge expense. If we have many intermittent tests running many times, then sometimes they will fail in all 25 runs and sometimes they will pass in all 25 runs. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
