On Wed, Jun 22, 2016 at 5:17 PM, Manish Goregaokar <manishsm...@gmail.com> wrote:
> On Wed, Jun 22, 2016 at 7:38 PM, Boris Zbarsky <bzbar...@mit.edu> wrote: > > > As a matter of dealing with test breakage and how it's resolved, do you > > also prefer doing a bunch of commits and only then running tests, and if > > they fail closing down the tree until it's all resolved? If not, how is > > this substantively different? I realize it's different in a practical > > sense because the costs are higher than normal per-commit testing, of > > course. > > > > > We don't close down the tree except for CI fires (and in the past, large > bitrot-prone rustups). We currently have tests run on every approved PR > (one at a time) before they are merged. We like for individual commits to > pass tests for bisecting to work well but it is not a hard requirement. If > the tests fail for a PR, it is taken out of the queue until it is fixed and > reapproved. In the meantime, the next PR in the queue is tested. We also > have travis perform a reduced set of tests on PRs so you can catch some > things that may cause it to fail before approving. Additionally, we have > the ability to "try" a PR so that it will be tested against the full suite > but not merged. We never have backouts, because master always is green. > I'm somewhat surprised about the optimism about keeping this model indefinitely going forward. I'm not aware of any truly large-scale project that has been able to do so, and I don't see why Servo would have any intrinsic advantages over these other projects in this regard. Hence, I think we'll have to make compromises on this sooner or later anyway. Whether that means roll-ups or something else, I'm not sure. Assuming this is true, it might make sense to factor it in when thinking about the integration with Gecko: if we go out of our way now to set up something that'd allow us to keep a setup we have to leave for other reasons in the not-too-distant future anyway, then that might not be worth it. Till _______________________________________________ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
