Hello everyone! Let's talk about when and how I would like us to use
"@bors-servo: retry" in our PRs.
* Which tests actually failed? Is each one filed in servo/servo as a
known intermittent failure?
* For the tests which are not previously known intermittents, what is
their output on this run? Are there any errors or panics in the log that
look related to the PR's changes?
* Is there anything in the test itself that looks like the PR's changes
could affect it? Pay particular attention to tests that report an
unexpected pass.
* Have issues been filed for each test which intermittently failed for
reasons that cannot be ascribed to this PR's changes? Are these issues
in the form "Intermittent [timeout|fail|crash|pass] for
/path/to/test.html" with the I-intermittent label applied and the
complete output of that test from the failing run copied and pasted?
Build logs disappear; github comments are forever.
* Has justification been provided in this "@bors-servo: retry" comment
for why a retry is necessary. Preferably this happens in the form of "*
#[issue number 1], #[issue number 2]".
I know this is more work than just slapping "@bors-servo: retry" in PR
and moving on with our lives. If there is a step in here that is slowing
us down, let's discuss ways we can make it quicker and easier. We're in
the middle of an upswing of intermittent timeouts, and I really want to
stay on top of this.
I've also recently noticed a number of instances of retrying PRs when
the logs show clear regressions through panics, etc. Please remain
vigilant about this and take the time to look at the actual test output,
rather than assuming any test failures are simply new intermittent
failures. It makes the queue run faster for everybody!
Cheers,
Josh
_______________________________________________
dev-servo mailing list
dev-servo@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-servo
