Hi all, During the discussion about splitting the main repo Ralph raised the problem that we don't have reliable integration tests.
Sure, most of our unit tests actually check the entire lifecycle of a logger context (which I think is a waste of testing time by the way), but these are single classloader Java SE deployments. What we don't test is: * how Log4j behaves in real world deployments: application servers, servlet containers, Spring Boot applications, OSGI containers. Yes, we have **some** tests inside containers, especially for OSGI, but the bootstrap procedure is very different from what real containers use. E.g. when we programmatically load `log4j-api` and `log4j-core` in our tests, everything works fine, but our users report that this does not work in an Eclipse RCP application, * how different components work together: e.g. is it possible to log both to Cassandra and Flume? I have looked a little bit around, what integration testing frameworks are out there: 1. Development on PAX Exam[1] (OSGI testing + some servlet testing) has as far as I can tell stopped, 2. There is a new OSGI Test[2] project, but I didn't test what it actually does, 3. Arquillian seems the most promising one with many container implementations[5] available, although we would need probably right some Log4j specific ones: e.g. a managed Tomcat container with Log4j artifacts in the right places, 4. Matt mentioned some time ago Testcontainers[6] that would allow us to test components against Docker images. AFAIK, the first three create an appropriate deployable artifact (OSGI bundle, WAR file) from our test class and deploy it to a container. I am not sure how Testcontainers work. Do you have any experience with these frameworks? Would it be useful to add a separate integration tests repo? Or just a Maven module? Or perhaps should we just add tests to the existing Maven modules? Piotr [1] https://github.com/ops4j/org.ops4j.pax.exam2 [2] https://github.com/osgi/osgi-test [3] http://arquillian.org/ [5] https://github.com/orgs/arquillian/repositories?q=arquillian-container [6] https://testcontainers.com/modules/
