Is the Talos sherriff going to confirm that the backout actually fixed the regression? If it didn't, can we reland?
On Thu, Dec 18, 2014 at 6:47 AM, jmaher <joel.ma...@gmail.com> wrote: > > Mozilla - 2015 Talos performance regression policy > > Over the last year and a half the Talos tests have been rewritten to be > more useful and meaningful. This means we need to take them seriously and > cannot just ignore real issues when we don't have time. This does not mean > we need to fix or backout every changeset that caused a regression. > > Starting in 2015, when a regression is identified to be related to a > specific changeset, the patch author will be ask for information via the > needinfo flag. We expect a response and reasonable dialog within 72 hours > (3 business days) of requesting information. If no response is given we > will backout the patch(es) in question and the patch author can investigate > when they have time and reland. > > Some requirements before requesting needinfo: > * On integration branches (higher volume), a talos sheriff will have > verified the root cause within 1 week of the patch landing > * a patch or set of patches from a bug must be identified as the root > cause. This can take place through retriggers on the tree or in the case > of many patches landing at once this would take place through a push to try > backing out the suspected patch(es) > * links in the bug to document the regression (and any related > regressions/improvements) > * if we are confident this is the root cause and it meets a 3% regression > threshold, then the needinfo request will mention that this policy will be > enforced > > Acceptable outcomes: > * A promise to attempt a fix at the bug is agreed upon, the bug is > assigned to someone and put in a queue. > * The bug will contain enough details and evidence to support accepting > this regression, we will mark it as wontfix > * It is agreed that this should be backed out > > Other scenarios: > * A bug related to the alert is not filed within 1 week of the patch > landing. This removes the urgency and required action. > * We only caught a regression at uplift time. There is a chance this > isn't easily determined, this will be documented and identified patch > authors will use their judgement to fix the bug > * Regression is unrelated to code (say pgo issue) - this should be > documented in the bug and closed as wontfix. > * When we uplift to Aurora or Beta, all regressions filed before the > uplift that show up on the upstream branch will have a needinfo flag set > and require action to be taken. > > > Please take a moment to look over this and outline any concerns you might > have. > > Thanks, > Joel > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
