shitikanth opened a new issue, #3356: URL: https://github.com/apache/maven-surefire/issues/3356
## Summary When all test methods in a JUnit 5 test class live inside `@Nested` inner classes (none at the outer level), the `.txt` surefire report for the outer class shows `Tests run: 0, Failures: 0, Errors: 0` — regardless of whether tests pass, fail, or error. The XML report for the same class correctly reflects the actual test count. ## Environment - `maven-surefire-plugin`: **3.5.5** - JUnit Jupiter: 5.11.4 - Java: 17 ## Steps to reproduce Minimal reproducer: https://github.com/shitikanth/surefire-nested-txt-report-reproducer ```java class OuterTest { @Nested class GroupA { @Test void test_one() { assertTrue(true); } @Test void test_two() { assertTrue(true); } } @Nested class GroupB { @Test void test_three() { assertTrue(true); } } } ``` ``` mvn test ``` ## Actual behaviour `target/surefire-reports/OuterTest.txt`: ``` Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 -- in OuterTest ``` `target/surefire-reports/TEST-OuterTest.xml`: ```xml <testsuite name="OuterTest" tests="3" errors="0" failures="0" .../> ``` ## Expected behaviour `OuterTest.txt` should report `Tests run: 3`, consistent with the XML. ## Impact - CI tooling or reporting tools that consume `.txt` files per class see 0 tests where there are actually N — results appear to silently vanish. - When `@BeforeEach` on the outer class throws, the XML shows errors but the TXT still shows 0, making errors invisible in the text report. ## Root cause (analysis from SUREFIRE-2194) `RunListenerAdapter` does not distinguish the outer class from its nested classes — each starts a new test set. `TestSetRunListener` does not support nested test sets, so the nested class's test set supersedes the outer class's, which closes with 0 tests. The XML reporter aggregates correctly across the hierarchy; the TXT reporter does not. ## Suggested fix Two approaches are viable: **Option 1 — Emit a separate TXT report per nested class**, mirroring how the XML reporter already produces `TEST-OuterTest$GroupA.xml` entries. Each nested class gets its own `.txt` file with an accurate count. The outer class `.txt` would then correctly show 0 (it has no tests of its own) and tooling would find the results in the nested class files. **Option 2 — Aggregate nested results into the top-level TXT report**, matching the current XML behaviour where all nested results roll up into `TEST-OuterTest.xml`. The outer class `.txt` would show the sum of all nested tests, giving a single accurate summary per top-level class. Option 2 keeps the TXT and XML in sync by design. Option 1 is more granular but requires tooling to follow the `$`-separated nesting to find all results. ## Relation to prior issues - **SUREFIRE-2194** (#2682, open since 3.1.2) — same root cause, different symptom: with tests at *both* outer and nested levels, results land in the wrong XML file. Our case (all tests nested, outer TXT shows 0 while XML is correct) is not covered there. - **SUREFIRE-2298** (#2601, fixed in 3.5.4 via PR #828) — fixed a related XML regression for Cucumber/ArchUnit. The TXT inconsistency was not addressed by that fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
