On Wed, Apr 03, 2013 at 04:59:31PM -0700, Jeff Hammel wrote:
> On 04/03/2013 04:44 PM, Joshua Cranmer 🐧 wrote:
> >On 4/3/2013 5:36 PM, L. David Baron wrote:
> >>On Wednesday 2013-04-03 17:31 -0400, Kartikaya Gupta wrote:
> >>>1. Take the latest green m-c change, commit your patch(es) on top of
> >>>it, and push it to try.
> >>>2. If your try push is green, flag it for eventual merge to m-c and
> >>>you're done.
> >>>3. If your try push is not green, update your patch(es) and go back
> >>>to step 1.
> >>This seems like it would lead to a substantial increase in
> >>build/test load -- one that I suspect we don't currently have the
> >>hardware to support. This is because it would require a full
> >>build/test run for every push, which we avoid today because many
> >>builds and tests get merged on inbound when things are behind. (I
> >>also don't feel like we need a full build/test run for every push,
> >>so it feels like unnecessary use of resources to me.)
> >Continuing in the vein of people posting ideas for "use less
> >infrastructure", an idea I had is this:
> >
> >Instead of running
> >{mochitest-*,reftest,crashtest,xpcshell,marionette} on every
> >single desktop platform on every inbound checkin, run them
> >round-robin. A given push would trigger mochitest on Linux,
> >reftest on mac, and then the next test would run reftest on Linux
> >and mochitest on Mac. Each push would still end up running the
> >full testsuite (modulo some tests that are only run on some
> >platforms) on some platform, and we would be using fewer test
> >resources to achieve that coverage. If most actual test bustage is
> >not platform-specific, this would give us generally sufficiently
> >good coverage for most checkins. I am not qualified to say,
> >however, if this is the case.
Another important question is what the net effect on machine load this
has is. I'd expect this would be a win over the coalescing we do today
but I'm not sure of that. If we reduce load significantly making it
somewhat slower to get full test results may well be acceptable because
we have many more cycles to spend on retriggers.
> If there was a way/guidelines to do this in some sensible way, or
> even a cultural norm like "reviewer comments on what tests to run",
> I'd give a +0.5 here fwiw
a smarter system than the one jcranmer is proposing certainly has merit
but its also significantly harder since it requires specifying what is
likely to be effected by each part of the tree and then keeping that up
to date.
Trev
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
