Hi, On Thu, Aug 04, 2016 at 12:45:50PM -0700, Bobby Holley wrote:
> One fundamental problem with precommit testing the precise commit to be > merged is that intermittents are more costly. You either have to hold up > the pipeline while you retrigger the intermittent job, or throw away all of > the green results you got and try again later. I'm not sure I understand. With the auto-rollup proposal, I don't see what green results there would be to throw away, as that one doesn't involve any speculative building... It's true though that frequent intemittent failures will considerably increase latency. For the pipelining proposal, I omitted the question of how to handle builds finishing out of order. I see two obvious options: one is to always land the commits in order, i.e. if a build of a later commit finished before a previous one, the result is held until the previous one lands. In this case, we would indeed dismiss successful results if a previous commit happens to fail. The other option however is not to wait, but rather to automatically land everything up to the latest successful commit, without waiting for the tests of the previous commits to finish. (So effectively it's a bit like a rollup.) This way we might in theory land commits that actually break things, if other commits coming later fix them again... However, this situation seems so obscure, that I don't think it's worth considering at all. Of course this would also potentially ignore intermittent failures in the unfinished builds -- but at the pre-commit testing phase, we actually *want* to ignore intermittent problems. One problem I *do* see is that in the original pipelining proposal, I said that if tests fail for one commit, all speculatively running tests for commits coming after it are immediately cancelled. In the face of frequent intermittent failures, we might want to actually let them run to completion instead, and if they succeed, land them anyways (along with the failed one) -- this way the redundancy of doing multiple speculative test runs becomes an asset. > At a higher level: the advantage of the proposed approach (3) is that jives > very well with the way TaskCluster and TreeHerder currently work, which > makes it a very tractable model upon which to align both mozilla-central > and servo/servo. I think we should start with that as a minimum-viable > implementation, and then tweak design and algorithms if there are obvious > improvements that we can envision. The problem here is that for people currently working with the mozilla-central approach, this is the baseline, and anything else would be optional improvements; but for people coming from Servo, full pre-commit testing is the baseline, and anything that goes back on this feels like a major regression... -antrik- _______________________________________________ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
