2012/1/19 Christopher Schultz <ch...@christopherschultz.net>: > All, > > I was testing 7.0.25 and "ant test" reports BUILD SUCCESSFUL but I > started looking at the TEST-*.txt files that are emitted and I was > wondering about a few things. > > First, I should probably be look at the bottom of the file for the junit > summary that looks like this: > > Testsuite: org.apache.tomcat.util.threads.TestLimitLatch > Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 2.545 sec > > Observing the "Failures: 0, Errors: 0" indicates that the tests have all > passed correctly, right? > > I'm asking because I can also see things like this: > > TEST-org.apache.catalina.connector.TestMaxConnections.APR.txt:INFO: > There were [4] passed requests and [2] connection failures
1. I was also puzzled. I tried to comment some similar cases, but the work is far from being complete. Especially I am puzzled by SEVERE messages. This is one case where I wrote a comment. If you look closely [1]: INFO: This test tries to create 10 connections to connector that has maxConnections='4'. Expect half of them to fail. [1] http://vmgump.apache.org/gump/public/tomcat-trunk/tomcat-trunk-test/gump_file/TEST-org.apache.catalina.connector.TestMaxConnections.BIO.txt.html Well, I see maybe I shall move that log statement. This test limits the count of simultaneous connections. Some of connections are rejected right away, but some will be caught in a OS' accept queue and time out. Both results are OK. That is in the sense of handling the limit, but user's experience will be different. In general to suppress certain SEVERE messages triggered by tests and printed by ErrorReportValve I think a custom ErrorReportValve might be needed. > > Obviously, that's an INFO line, but it does indicate a "failure" of some > kind. > > There are also some log lines like this: > > TEST-org.apache.catalina.startup.TestListener.NIO.txt:SEVERE: Context > [/] startup failed due to previous errors > > Does that merely indicate that the test itself caused a failure, and > that the failure-to-startup was intentional? 2. That is likely a standard message, intentionally triggered by the test. > Similarly: > > TEST-org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.APR.txt:org.apache.catalina.tribes.ChannelException: > Send failed, and sender is disconnected. Not retrying.; Faulty > members:tcp://{127, 0, 0, 1}:4005; > > Again, all test files say "Failures: 0, Errors: 0" so I guess everything > is okay. It's just tough to see those log lines without asking. > 3. That is standard message. It is expected and triggered by the test. But personally I lack complete understanding of tribes tests. Some of them fail occasionally (triggering Gump or Buildbot e-mails) and I do not know what to do. 4. I have seen several cases where Tomcat shutdown was failing (generating messages), but JUnit tests were not failing. Maybe that is because of different threads. I do not know how improve that. I personally always review results and stdout for "NullPointerException". NPEs are unexpected. Best regards, Konstantin Kolinko --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
