Hi all,
During the discussion about splitting the main repo Ralph raised the
problem that we don't have reliable integration tests.
Sure, most of our unit tests actually check the entire lifecycle of a
logger context (which I think is a waste of testing time by the way),
but these are single classloader Java SE deployments. What we don't
test is:
* how Log4j behaves in real world deployments: application servers,
servlet containers, Spring Boot applications, OSGI containers. Yes, we
have **some** tests inside containers, especially for OSGI, but the
bootstrap procedure is very different from what real containers use.
E.g. when we programmatically load `log4j-api` and `log4j-core` in our
tests, everything works fine, but our users report that this does not
work in an Eclipse RCP application,
* how different components work together: e.g. is it possible to log
both to Cassandra and Flume?
I have looked a little bit around, what integration testing frameworks
are out there:
1. Development on PAX Exam[1] (OSGI testing + some servlet testing)
has as far as I can tell stopped,
2. There is a new OSGI Test[2] project, but I didn't test what it actually does,
3. Arquillian seems the most promising one with many container
implementations[5] available, although we would need probably right
some Log4j specific ones: e.g. a managed Tomcat container with Log4j
artifacts in the right places,
4. Matt mentioned some time ago Testcontainers[6] that would allow us
to test components against Docker images.
AFAIK, the first three create an appropriate deployable artifact (OSGI
bundle, WAR file) from our test class and deploy it to a container. I
am not sure how Testcontainers work.
Do you have any experience with these frameworks? Would it be useful
to add a separate integration tests repo? Or just a Maven module? Or
perhaps should we just add tests to the existing Maven modules?
Piotr
[1] https://github.com/ops4j/org.ops4j.pax.exam2
[2] https://github.com/osgi/osgi-test
[3] http://arquillian.org/
[5] https://github.com/orgs/arquillian/repositories?q=arquillian-container
[6] https://testcontainers.com/modules/
