arimu1 opened a new pull request, #3413: URL: https://github.com/apache/maven-surefire/pull/3413
## Summary Fixes #3412 Since Surefire 3.5.5, exceptions thrown from JUnit `@AfterAll` / `@AfterClass` (class-level teardown) were no longer reported as errors. The suite totals showed `errors="0"`, the teardown failure was misclassified as a flake (`flakes="1"`), and with `rerunFailingTestsCount > 0` already-passing test methods were re-executed while the build could stay green. This regressed when BeforeAll flake handling treated any class-level failure with a null / `initializationError` name as flaky whenever sibling test methods had succeeded — which is exactly the AfterAll pattern (tests pass, then teardown fails). ### Fix In `RunListenerAdapter`, distinguish setup vs teardown class-level failures: | When | Report name | Rerun? | Classification | |------|-------------|--------|----------------| | Class fails before any test method succeeds | `initializationError` | yes (existing BeforeAll path) | may be FLAKE if a later rerun succeeds | | Class fails after at least one test method succeeded | `executionError` | **no** | always **ERROR** | Reporter special-casing for flaky BeforeAll is limited to `initializationError` (and the legacy `"null"` method name string). `executionError` goes through the normal error path in `DefaultReporterFactory` and `StatelessXmlReporter`. ### Manual verification (Temurin 21, installed SNAPSHOT plugin) **@AfterAll always fails:** ``` Tests run: 3, Failures: 0, Errors: 1 (class) Tests run: 4, Failures: 0, Errors: 1 (suite) XML: tests="3" errors="1" flakes="0" name="executionError" BUILD FAILURE ``` **@AfterAll + `rerunFailingTestsCount=2`:** passing methods run **once** (not 3×); still `errors="1"`. **@AfterClass (JUnit 4 / vintage):** same — `executionError`, `errors="1"`. **@BeforeAll always fails:** still `initializationError`, `errors="1"` (unchanged). ## Checklist - [x] Each commit in the pull request should have a meaningful subject line and body. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Run `mvn clean install` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] You have run the integration tests successfully (`mvn -Prun-its clean install`). (Focused unit tests + manual IT projects exercised; full `-Prun-its` left to CI.) - [x] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) ## Tests - `DefaultReporterFactoryTest` (incl. `testAfterAllFailureRemainsErrorNotFlake`) — **5/5** - `RunListenerAdapterTest` (incl. AfterAll naming / no-rerun) — **34/34** - `JUnitPlatformProviderTest` (incl. `shouldErrorClassOnAfterAll`) — **35/35** - `StatelessXmlReporterTest` — **12/12** - New ITs: `JUnit5FailingAfterAllIT`, `JUnit4FailingAfterClassIT` (+ resources) - Manual nested projects with SNAPSHOT surefire as above ## Related - Regression introduced while handling BeforeAll flakes (`b4a53de`) - Prior attempt documenting current broken behavior: #3338 (ITs only; this PR fixes the behavior) - Empty-name fix for container failures: #3329 (`initializationError` naming) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
