To build off this idea, I'd like a "run-until-failure" mode (with an upper limit, of course) on try itself. I don't want to spend N+ hours spinning my CPU locally to repro an intermittent. I also don't want to wait until a build is done to press the retrigger button 40 times.
My blue-sky wish would be to push a run-until-failure with RR try job, and download the failing replays the next day to debug locally. On Thu, Mar 12, 2015 at 4:34 PM, Seth Fowler <s...@mozilla.com> wrote: > > > On Mar 12, 2015, at 4:17 PM, Gijs Kruitbosch <gijskruitbo...@gmail.com> > wrote: > > It'd be better if we could more easily get more information about > failures as they happened on infra (replay debugging stuff a la what roc > has worked on, or better logs, or somehow making it possible to > remote-debug the infra machines as/when they fail). > > > > ~ Gijs > > Yes! > > I guess (but don’t know for sure) that recording RR data for every test > that gets run might be too expensive. But it’d be nice if we were able to > request that RR data be recording for *particular* tests in the test > manifest. Probably we should make it possible to record it unconditionally, > and to record it only if the test failed. I can see using this in two ways: > > - When I’m investigating a failure, I can push a commit that records RR > data for the test I’m concerned about. > > - We can annotate rare intermittent failures to record RR data whenever > they fail. This obviously means that the tests must be run under RR all the > time, but at least we won’t have to waste storage space on RR data for the > common, passing case. This would be a huge help for some hard-to-reproduce > intermittent failures. > > I’m sure there are a lot of gotchas here, but in an ideal world these > features would be a huge help. > > - Seth > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > -- shu _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
