Testcontainers is really neat, but it does require Docker, so I’m not sure how to best use it in CI (particularly outside Linux). Arquillian is a classic option that’s been around for as long as I’ve been using Java for server-side anything, though I never really figured out how to use it (it seems similar to Pax Exam, both of which confuse me). Finding how to reuse Maven modules in these test projects is the tricky part in my experience, though maybe that’s fairly easy to do these days.
> On Sep 10, 2023, at 2:51 PM, Piotr P. Karwasz <piotr.karw...@gmail.com> wrote: > > Hi all, > > During the discussion about splitting the main repo Ralph raised the > problem that we don't have reliable integration tests. > > Sure, most of our unit tests actually check the entire lifecycle of a > logger context (which I think is a waste of testing time by the way), > but these are single classloader Java SE deployments. What we don't > test is: > > * how Log4j behaves in real world deployments: application servers, > servlet containers, Spring Boot applications, OSGI containers. Yes, we > have **some** tests inside containers, especially for OSGI, but the > bootstrap procedure is very different from what real containers use. > E.g. when we programmatically load `log4j-api` and `log4j-core` in our > tests, everything works fine, but our users report that this does not > work in an Eclipse RCP application, > * how different components work together: e.g. is it possible to log > both to Cassandra and Flume? > > I have looked a little bit around, what integration testing frameworks > are out there: > > 1. Development on PAX Exam[1] (OSGI testing + some servlet testing) > has as far as I can tell stopped, > 2. There is a new OSGI Test[2] project, but I didn't test what it actually > does, > 3. Arquillian seems the most promising one with many container > implementations[5] available, although we would need probably right > some Log4j specific ones: e.g. a managed Tomcat container with Log4j > artifacts in the right places, > 4. Matt mentioned some time ago Testcontainers[6] that would allow us > to test components against Docker images. > > AFAIK, the first three create an appropriate deployable artifact (OSGI > bundle, WAR file) from our test class and deploy it to a container. I > am not sure how Testcontainers work. > > Do you have any experience with these frameworks? Would it be useful > to add a separate integration tests repo? Or just a Maven module? Or > perhaps should we just add tests to the existing Maven modules? > > Piotr > > [1] https://github.com/ops4j/org.ops4j.pax.exam2 > [2] https://github.com/osgi/osgi-test > [3] http://arquillian.org/ > [5] https://github.com/orgs/arquillian/repositories?q=arquillian-container > [6] https://testcontainers.com/modules/
