On Thursday, December 29, 2016 at 2:04:28 PM UTC-5, Boris Zbarsky wrote: > Andrew, thank you for catching this! This sounds a _lot_ like > https://bugzilla.mozilla.org/show_bug.cgi?id=1321127 in terms of cause > and effects.
Yes, that bug is a similar failure. In both cases we should have had some sort of automated testing to catch regressions in mozharness' log parsing. When I start working on trying to get this stood up, I will make sure we check all of test failures, assertions, leaks and crashes across all our harnesses. > Have we done any sort of audit to see whether any other tests got broken > by the structured logging changes? Had we done such an audit after bug > 1321127 was fixed? Once bug 1321127 was fixed, any other tests that were broken would turn the job orange so would have prevented the fix from landing. It's possible (though highly unlikely), that a failure was "accidentally fixed" on central but not aurora before we detected the problem. I can push the fix to aurora too just to be sure. > Have the bugs in that dependency tree you link above been nominated for > tracking for the relevant branches? I _think_ the leak detection has > been broken since sometime in the 51 timeframe at least for some test > harnesses (based on the target milestone on bug 1261199, for example), > so presumably we need to backport the fix and figure out which leak bugs > are happening on aurora and beta, then set tracking flags, etc, right? Yes, there are a couple leaks on the 52 branch and their tracking flags have been set accordingly. I haven't noticed any on 51 yet, though it could be possible. Joel and I are still working through the list to better triage them, but as they are getting fixed we are double checking that they aren't on aurora/beta as well. > > Again, thank you for catching this. > > -Boris _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
