I am new to the use of the try build system and so this is not a suggestion informed by much experience with this particular setup, but time spent in other environments has shown me that running these sorts of test builds on all platforms can sometimes be quite useful.
I'd suggest a variation on this idea: all-platform try builds should automatically be scheduled to only one platform initially, that with the fastest turnaround and lowest load. If there are issues on that platform, no further platforms are scheduled. If that platform is all green, the remaining platforms are scheduled in parallel as happens now. This seems to me to offer the best of all possible worlds: we fail early in the case where there are problems, but we also do our best to flush out issues that may not have been caught on the initially chosen platform. Thanks, - Seth On Sep 28, 2012, at 3:42 AM, Bobby Holley <bobbyhol...@gmail.com> wrote: > Hi All, > > I'd like to propose that we discourage the use of all-platform try builds. > > I cringe a little bit every time I see a full column of orange on a try > push. More often than not, it means that we spent an order of magnitude > extra compute time verifying that, indeed, the push caused a given test to > fail on every single platform. > > In my experience, a debug-only build on a single platform (say, linux64) > gives me about 80% of the information that I get from |-b do -p all|. If a > push goes entirely green on a given platform, the chances are decent that > it will go green everywhere. If it goes orange, our infrastructure is > spared the formality of reproducing the failure in every configuration. If > the orange is questionably intermittent, the job can be retriggered > (several times) with a single click of the '+' button on the build summary. > > Single-platform builds certainly won't catch those pesky WinXP-only > browser-chrome oranges. But that's what mozilla-inbound is for. The social > contract of inbound doesn't, in my opinion, require that the developer > painstakingly verify correctness in every configuration. Rather, developers > are just expected to be reasonably confident that the push won't turn the > entire tree bright orange. And for most changes, a green push on a single > platform gives such confidence. > > There are of course exceptions, especially with massive landings (CPG, > DLBI), and various large widget-y or graphics-y changes. But unless the > landing schedule is tight enough that 2-3 hours really matters, I'd > encourage developers to do a single-platform push first in order to catch > the low-hanging fruit. > > If everyone makes an effort in this department, it will make a significant > impact on infrastructure load, which in turn makes life easier for > everyone, especially our valiant sheriffs. At least I'd think so. I'm not > an expert in this stuff, so if there's some glaring flaw with this approach > by all means raise it. > > On that note, it would be nice if trychooser could implement some kind of > |-p any| syntax that would select a platform with the fastest turnaround > time and lowest load. This would save developers from manually doing such > round-robin scheduling between linux, linux64, and macosx64 (which, in my > experience, produce the fastest runs). > > Thoughts? > bholley > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
