I seem to recall gps saying, a few years ago, that the cost per push or
Try run was about $36. It would be good to know the current cost so
developers can feel more comfortable spending Mozilla money on
"unnecessary" Try runs before landing.
On 3/10/2017 6:47 AM, Andrew Halberstadt wrote:
I don't have any data to back this up, but my suspicion is that a large
percentage
of backouts had try runs, but said try runs didn't run the jobs that failed
and caused
the backout. Imo, these kinds of backouts are (more) acceptable because it
means
the developer was trying to avoid doing a full try run, a practice that
should be
cheaper overall in the long run (and if done properly).
For example, you could either:
A) Do a full try run then push, almost guaranteeing you won't be backed
out. But
then you run every job twice and take longer to complete your bug, a
significant
cost.
B) Do a partial try run, running X% of the jobs yielding a Y% chance of
being
backed out.
There's some sweet spot between no try run, and try: -all which is the most
cost
effective (both in terms of AWS bill and developer time).
That being said, I think this is an issue of tools rather than policy.
Things like being
smarter about running jobs based on files changed and improving interfaces
to
selecting jobs on try, should help with backout rates. But the single
biggest thing we
could do is getting rid of integration branches altogether (and instead get
autoland to
merge directly from try). In this world, backouts would hardly even exist
anymore.
I believe we're already headed in this direction, albeit slowly.
-Andrew
On Fri, Mar 10, 2017 at 8:55 AM, David Burns <dbu...@mozilla.com> wrote:
I went back and did some checks with autoland to servo and the results are
negligible. So from 01 February 2017 to 10 March 2017 (as of sending this
email). I have removed merge commits from the numbers.
Autoland:
Total Servo Sync Pushes: 152
Total Pushes: 1823
Total Backouts: 144
Percentage of backouts: 7.8990674712
Percentage of backouts without Servo: 8.61759425494
Mozilla-Inbound:
Total Pushes: 1472
Total Backouts: 166
Percentage of backouts: 11.277173913
I will look into why, with more pushes, is resulting in fewer backouts. The
thing to note is that autoland, by its nature, does not allow us to fail
forward like inbound without having to get a sheriff to land the code.
I think, and this is my next area to investigate, is the 1 bug per push
(the autoland model) could be helping with the percentage of backouts being
lower.
David
On 7 March 2017 at 21:29, Chris Peterson <cpeter...@mozilla.com> wrote:
On 3/7/2017 3:38 AM, Joel Maher wrote:
One large difference I see between autoland and mozilla-inbound is that
on
autoland we have many single commits/push whereas mozilla-inbound it is
fewer. I see the Futurama data showing pushes and the sheriff report
showing total commits.
autoland also includes servo commits imported from GitHub that won't
break
Gecko. (They might break the linux64-stylo builds, but they won't be
backed
out for that.)
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
