------- Comment #10 from amylaar at gcc dot gnu dot org 2006-12-19 12:57 ------- (In reply to comment #9) > OK, so it looks like this is not a testsuite problem: the problem really is > with the program being tested.
It is also a testsuite problem when the tests are randonly passing or failing; such tests are not suitable as such in an automated testsuite. I suppose you could write a wrapper to run them several times to get a reasonable high confidence that the test generally passes - if the test does not take too long - and maybe the probabilities can also be enhanced by forking specifically designed other programs to perturb the load, but unless/ until such infrastructure is put in place, these randomly failing tests make it effectively impossible to pass a regression test. I've redone to tests on my baseline where the test passed during the automated test, and the results are again random: [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ /home/amylaar/bld/2006-11-27/i686/gcc/gcj -B/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/ -B/home/amylaar/bld/2006-11-27/i686/gcc/ --encoding=UTF-8 -C -I/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite/../libgcj-4.3.0.jar -g /home/amylaar/bld/2006-11-27/srcw/libjava/testsuite/libjava.lang/PR18699.java -d /home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ export ld_library_path=.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ export LD_LIBRARY_PATH=.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:/home/amylaar/bld/2006-11-27/i686/./bfd/.libs:/home/amylaar/bld/2006-11-27/i686/./prev-bfd/.libs:/home/amylaar/bld/2006-11-27/i686/./opcodes/.libs:/home/amylaar/bld/2006-11-27/i686/./prev-opcodes/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libstdc++-v3/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libmudflap/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libssp/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libgomp/.libs:/home/amylaar/bld/2006-11-27/i686/./gcc:/home/amylaar/bld/2006-11-27/i686/./prev-gcc [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? Exception in thread "Thread-2" java.lang.NullPointerException at PR18699.update(PR18699.java:34) at java.util.Observable.notifyObservers(Observable.java:171) at PR18699.run(PR18699.java:26) at java.lang.Thread.run(Thread.java:682) 1 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? Exception in thread "Thread-1" ^[[Ajava.lang.NullPointerException at PR18699.update(PR18699.java:34) at java.util.Observable.notifyObservers(Observable.java:171) at PR18699.run(PR18699.java:26) at java.lang.Thread.run(Thread.java:682) 1 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? Exception in thread "Thread-2" ^[[A java.lang.NullPointerException at PR18699.update(PR18699.java:34) at java.util.Observable.notifyObservers(Observable.java:171) at PR18699.run(PR18699.java:26) at java.lang.Thread.run(Thread.java:682) 1 [EMAIL PROTECTED]:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij -classpath .:../libgcj-4.3.0.jar PR18699;echo $? 0 -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119
