On Thu, May 11, 2017 at 10:05 AM, <dmitch...@mozilla.com> wrote: > Background: > https://bugzilla.mozilla.org/show_bug.cgi?id=1359942 > > As jobs move to taskcluster, we have an improved opportunity to do some > smarter scheduling of what jobs to run on what sort of push. Of course, > it's a thorny subject: optimizing away a task that should run may let a bad > push show green, while a subsequent push bears responsibility for the > orange it introduces. > > One of the more common expectations is that pushes that only change a > directory affecting one platform should not cause other platforms' tasks to > run. > > In the bug above, I have proposed a method of identifying pushes > "affecting" a particular platform, and Greg has raised some concerns about > the generality of my solution. I'm happy to generalize, but I would like > to keep the process in motion rather than let the perfect be the enemy of > the good. > > To that end, I'd like some further feedback on implementing this sort of > optimization support. > > If there's sufficient interest, then this is probably something we could > set up a time to talk about in SFO in June. >
I still owe a proper reply to everything in this thread. But as I'm preparing to send out another Firefox developer survey, I'm looking at the old one we conducted and there are some results that seemingly justify doing work to intelligently run things based on what changed. One of the questions on the last survey was "Thinking of running automated tests, rank the following potential improvements in terms of their impact on your productivity." "Determine and run relevant tests based on what source files have been modified" was one of the most wanted improvements - right up there with "make try runs really fast so I can effectively iterate on automated tests using try instead." Another question was "Rank the following issues according to how they impact your ability to debug tests locally." "Tests take too long to run / stop me from using my computer" was one of the "most impact" elements. Although I concede the "stop me from using my computer" bit could be contributing a bit of weight, since front-end developers were 2:1 more likely than platform developers to answer "most impact." This could be attributed to many mochitests requiring focus and thus rendering your machine unusable. If you want, I can try to tease out the developer impact of running fewer tests in the next survey.
_______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
