On Mon, Sep 28, 2015 at 2:12 AM, Pavel Labath <lab...@google.com> wrote:
> Hi, > > Interesting results. We were discussing the same thing last week. I > was somewhat skeptical to the ideal as I am afraid of increased > flakyness -- LLDB has hardcoded timeout values in a lot of places, and > with increased cpu contention, we might start to see this code failing > because the other side was slower than expected. > Yeah, when I first did this a while back (just the parallel piece, not the speedup of it), I think it uncovered a ton of races/flakiness. Much of that has been addressed, but nothing like forcing more contention to work out more of them. > > I have now tried running the test suite with a higher number of > threads and it seems it working quite alright, so I think we can try > that. I'll watch the buildbots for signs of increased flakyness. FWIW, > it did not speed up the tests for me at all, but I guess that's > expected, as it ran in 90 seconds even before that, and the limiting > factor there probably is the longest test. > > Yeah, all those lengthy debugserver/lldb-server tests I wrote a while back tend to have a long tail, and I'm sure you folks have only extended them since then. I have some thoughts on improving the job scheduling in the future (finer grain) but not in the short term. > As for the timeout, I have set the value to 4 minutes, based on the > formula: 2 * (number of seconds when I first started noticing flaky > tests in the slowest configuration). The slowest configuration here > was running the test suite on a debug build of lldb and using debug > ToT clang to compile the inferiors (believe it or not, this makes a > big difference). So there is still some leeway here, but if you're > gonna reduce the default, please make sure it is enough for the > slowest test configuration also. For faster configurations you can > always override the timeout manually. > > pl > Okay. I'm not planning on touching the timeout values at this point, just noting that they may be worth messing with in the future. I'd like to add a very lightweight stats database (that's already too heavy a word, but conveys the idea) that can be arranged to persist across builds (that last part specifically for buildbots). With just a tiny bit of historical data when available, job scheduling to reduce total time of test run could be improved. And with some finer grain scheduling which I plan to look into mid-term via a mechanism to prevent a single test method hangup from knocking out the rest of the test methods in the same run, we could do even better yet. But I'll bring this up when I get closer to having some time to work on it. (And that's not yet!) -- -Todd
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
