Re: improving testsuite runtime

2008-09-21 Thread Joern Rennecke
On Fri, Sep 19, 2008 at 04:08:50PM -0700, Janis Johnson wrote: > Would it work for you to have a check-init target to set up site.exp > and whatever else might be needed, a check-fini target to wrap up > the results, and multiple targets that you can invoke separately in > between those? A top-lev

Re: improving testsuite runtime

2008-09-19 Thread Janis Johnson
On Fri, 2008-09-19 at 18:32 +0100, Joern Rennecke wrote: > > I think 'make -j' is the way to go, since it lets the user easily > > control the amount of parallelism. > > As I said before, make -j is a complete non-starter for me, as it restricts > the paralelism to a single machine and thus would

Re: improving testsuite runtime

2008-09-19 Thread Joern Rennecke
> I think 'make -j' is the way to go, since it lets the user easily > control the amount of parallelism. As I said before, make -j is a complete non-starter for me, as it restricts the paralelism to a single machine and thus would actually reduce the parallelism from what I have now with multilibs

Re: improving testsuite runtime

2008-09-19 Thread Tom Tromey
> "Ben" == Ben Elliston <[EMAIL PROTECTED]> writes: Ben> Do you think that the current order of .exps should be preserved Ben> in the resultant .sum and .logs? I personally don't have a use for this. I just think that the order ought to be stable across checks of the same build. Tom

Re: improving testsuite runtime

2008-09-18 Thread Ben Elliston
> Do people still use compare_tests? Talking with Janis, she mentioned that > it wasn't multilib (ie, RUNTESTFLAGS="--target_board=unix'{-m32,-m64}') > compatible, but that test_summary was. It's what I've been using to > compare two runs. I have used compare_tests for a long, long time. Ben P

Re: improving testsuite runtime

2008-09-18 Thread Peter Bergner
On Fri, 2008-09-19 at 09:41 +1000, Ben Elliston wrote: > On Thu, 2008-09-18 at 10:44 -0600, Tom Tromey wrote: > > Yeah, this seems necessary. Ideally the order ought to be stable, too. > > Do you think that the current order of .exps should be preserved in the > resultant .sum and .logs? I guess

Re: improving testsuite runtime

2008-09-18 Thread Ben Elliston
> But stability within a given revision of the testsuite I think would be > almost essential. Oh, of course :-) Ben

Re: improving testsuite runtime

2008-09-18 Thread David Daney
Ben Elliston wrote: On Thu, 2008-09-18 at 10:44 -0600, Tom Tromey wrote: Ben> So, I guess my question is: what now? What do people feel would be Ben> required to make this usable? I assume that the most pressing thing Ben> would be to have the build system fold the various .log and .sum files

Re: improving testsuite runtime

2008-09-18 Thread Ben Elliston
On Thu, 2008-09-18 at 10:44 -0600, Tom Tromey wrote: > Ben> So, I guess my question is: what now? What do people feel would be > Ben> required to make this usable? I assume that the most pressing thing > Ben> would be to have the build system fold the various .log and .sum files > Ben> together

Re: improving testsuite runtime

2008-09-18 Thread Janis Johnson
On Thu, 2008-09-18 at 21:37 +1000, Ben Elliston wrote: > While waiting on testsuites this week, I finally snapped and spent some > time looking at how to speed up the testsuite. > So, I guess my question is: what now? What do people feel would be > required to make this usable? I assume that the

Re: improving testsuite runtime

2008-09-18 Thread Tom Tromey
> "Ben" == Ben Elliston <[EMAIL PROTECTED]> writes: Ben> Using this script and some minor gcc/Makefile.in hacks, I ran the entire Ben> testsuite in 30% of the current time for a parallel-languages make Ben> check. Awesome. Ben> So, I guess my question is: what now? What do people feel would

Re: improving testsuite runtime

2008-09-18 Thread Joern Rennecke
For arc-elf32, I only want to run C and C++ tests, so the runtime of fortran tests is irrelevant for this purpose. On the other hand, I run the tests eight-way multilibbed. Currently, I run the check-gcc on eight hosts (or execution slots on multicore servers), and the check-g++ tests on eight othe

improving testsuite runtime

2008-09-18 Thread Ben Elliston
While waiting on testsuites this week, I finally snapped and spent some time looking at how to speed up the testsuite. I did some experiments and collected data on the normalised runtimes of each .exp test script. I sorted them in descending order and these are the top offenders: