On Wed, Nov 17, 2010 at 6:55 AM, Stefan Behnel <[email protected]> wrote: > Robert Bradshaw, 17.11.2010 09:59: >> 1. The web site >> 5. Buildbot >> >> In terms of the website itself, I think we should keep hosting that >> (for maximum flexibility, and it's easy to administer) > > +1 > > >> and there >> isn't much of an option for 5 (though steps should be taken to reduce >> its load). > > I'm a big fan of timely commit feedback.
Me too, the build bot has been great. > Hudson is already down to 6 build > threads, so it takes a while now to finish the test suites. Also, the less > parallel builds we run, the longer it disturbs other programs on the same > machine. Yes, though it disturbs them less. I think we'll be able to bump it past 6 threads in the long run, it's just that things have been pretty bad lately. > Could you describe a little where you think the current pain points are? Is > it more about the number of builds per day or the overall load during a > single build? I/O or CPU load? My impression is that the /scratch/hudson > mount point doesn't have very fast I/O, which lets the builds take longer > than necessary. I/O is a big pain point. /scratch/hudson isn't very fast IO, but it's faster than the nsf mount. CPU peaks pretty badly too. > A problem that I see is that Hudson only checks for changes every 10 > minutes in order to keep the load of the (fairly slow) Cython VM > acceptable. So if a new commit arrives shortly after triggering the builds, > it will continue to run the tests for 10 minutes before it starts over and > does it all again. I tried changing that with the commit hook script but > didn't get a response. I looked and wasn't sure how to do it, contacted Harald Schilly (the de-facto admin) and he wasn't sure, and then we started thinking about migration, so this got put on hold. The current system is particularly bad as you or I happen to push just frequently enough to keep 12+ threads going at full tilt for the entire time we're working on Cython. > I don't know if bitbucket supports commit hooks, BTW. > > One way to get relief is to reduce the currently 9 jobs that all query the > cython-devel repo for changes. I can try to change that by adding a > dedicated source build job that does the polling and triggers the real > build jobs that just download the sdist with no hg interaction. That moves > the hg change history one step further away from the build result on the > Hudson job page, but that won't hurt anyone, I guess. I was actually thinking the exact same thing. Sdist is actually rather io-intensive, but I'd imagine all jobs pulling locally (disk-to-disk) triggered a single job that pulls from the web interface would be a big improvement. Two other things that might help out would be (1) doing much of the building on boxen's RAM disk and (2) having a cascading system where, for example, 2.3 and 2.6 were fired off, and only if all their tests passed would 2.4-2.6 run. (It seems like there's a lot of wasteful testing going on for the non-version specific failures.) - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
