On 2014-08-20, 10:58 AM, Ed Morley wrote:
On 19/08/2014 21:55, Benoit Girard wrote:
I completely agree with Jeff Gilbert on this one.
I think we should try to coalesce -better-. I just checked the current
state of mozilla-inbound and it doesn't feel any of the current patch
really need their own set of tests because they're are not time
sensitive or sufficiently complex. Right now developers are asked to
create bugs for their own change with their own patch. This leads to a
lot of little patches being landed by individual developers which
seems to reflect the current state of mozilla-inbound.
Perhaps we should instead promote checkin-needed (or a similar simple)
to coalesce simple changes together. Opting into this means that your
patch may take significantly longer to get merged if it's landed with
another bad patch and should only be used when that's acceptable.
Right now developers with commit access are not encouraged to make use
of checkin-needed AFAIK. If we started recommending against individual
landings for simple changes, and improved the process, we could
probably significantly cut the number of tests jobs by cutting the
number of pushes.
I agree we should try to coalesce better - however doing this via a
manual "let's get someone to push a bunch of checkin-needed patches in
one go" is suboptimal:
1) By tweaking coalescing in buildbot & pushing patches individually, we
could get the same build+test job per commit ratio as doing
checkin-neededs, but with the bonus of being able to backfill jobs where
needed. This isn't possible when say 10-20 checkin-neededs are landed in
one push, since our tooling can only trigger (and more importantly
display the results of) jobs on a per push level.
2) Tooling can help make these decisions much more effectively and
quickly than someone picking through bugs - ie we should expand the
current "only schedule job X if directory Y changed" buildbotcustom
logic further.
3) Adding a human in the workflow increases r+-to-committed cycle times,
uses up scarce sheriff time, and also means the person who wrote the
patch is not the one landing it, and so someone unfamiliar with the code
often ends up being the one to resolve conflicts. We should be using
tooling, not human cycles to lands patches in a repo (ie the
long-promised autoland).
Another approach to this would be to identify more patterns that allow
us to skip some jobs. We already do this for very simple things
(changes to a file in browser/ won't trigger b2g and Android builds, for
example), but I'm sure we could do more. For example, changes to files
under widget/ may only affect one platform, depending on which
directories you touch. Another idea that I have had is adding some
smarts to make it possible to parse the test manifest files, and
recognize things such as skip-if, to figure out what platforms a test
only change for example might not affect, and skip the builds and tests
on those platforms.
One thing to note is that there is going to be a *long* tail of these
types of heuristics that we could come up with, so it would be nice to
try to only address the ones that provide the most benefits. For that,
someone needs to look at the recent N commits on a given branch and
figure out what jobs we _could_ have safely skipped for each one.
Cheers,
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
