Hello Surefire experts! In the ArgoUML project the JUnit tests could be improved. Nevertheless we have been running them more or less nightly for more than 6 years. They run from ant with fork=yes and forkMode=perTest.
To modernize this I am attempting to run the same tests from a Hudson+Maven setup instead. In the first attempt I didn't find how to make the tests run each in its own jvm so almost half of the tests failed (901 fails out of 2014 same tests with and without coverage). Last night I finally understood how to make Surefire fork a jvm for each test (<forkMode>always</forkMode>) and the set the java.awt.headless and some other properties and managed to get 439 tests to work (there is more work to be done in this area). The problem I noticed with the setup is that running the tests as forking per test is really slow. Because of some configuration issues, paths not set up correctly, I noticed that the suite() methods were called for every test case for every fork. We have a handful of tests that use the suite() method. These does some file parsing and clever searches and that time is well spent when running these specific tests. But for most tests, this is pointless. I don't even see the point in loading tests that are not supposed to be run in each fork. When I looked at how it performed each jvm consumed about 1 minute and 15 seconds of CPU while the test reported time spent of a few seconds only. (This was on my 400MHz build host. On my 3GHz desktop machine it takes only 45 seconds of CPU). I get the feeling that something in the setup (Maven, Surefire, Junit 4.8.1, old junit3 test cases) goes through all existing test cases for every fork and run the suite() methods. The entire build took 8 hours instead of 3 that I had expected (it took 2 hours before). How do I turn this off? How do I get each fork to just load and run the test case it is supposed to run? The hudson build is at: http://closettop.homelinux.org:8080/job/maven-build/ . The source (including poms) can be retrieved using svn from http://argouml.tigris.org/svn/argouml/trunk or browsed at http://argouml.tigris.org/source/browse/argouml/trunk/. The problem is in the argouml-app subsystem. As you can see, if you attempt mvn test, the text "Skipping 15 abstract elements" is written once per fork from within suite() in the org.argouml.model.TestAgainstUmlModel class. /Linus
