Bing Xu created SUREFIRE-2284:
---------------------------------
Summary: Failures or errors in JUnit @BeforeAll will cause
Surefire to throw an exception even if the test succeeds in a rerun.
Key: SUREFIRE-2284
URL: https://issues.apache.org/jira/browse/SUREFIRE-2284
Project: Maven Surefire
Issue Type: Bug
Components: Maven Surefire Plugin
Reporter: Bing Xu
Hi again from Atlassian
*Bug description*
If a Test fails (either *fails* (via an assertion or directly calling fail) or
throws an {*}error{*}) in the {{@BeforeAll}} method, the interaction with the
{{rerunFailingTestsCount}} property doesn't work properly. Even if the test
succeeds in a re-run, the Surefire Mojo will throw an exception and fail the
build.
*Scenario/Steps to reproduce*
A test fails or throws an exception (error) in the @BeforeAll method on an
initial run (Run #1).
And then succeeds in a subsequent execution (Run #2).
*Current behaviour*
The surefire plugin will throw an exception because of the failure/error in the
initial run of the test class.
*Expected behaviour:*
The build should pass if a subsequent rerun of the test and no exception should
be thrown.
*Root cause*
The root cause seems to be because surefire has logic to merge the result of
multiple runs of the same test inside the following class:
{code:java}
maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/DefaultReporterFactory.java{code}
This class does so by matching test method stats with the same
testClassMethodName. This does not work when the test fails in @BeforeAll
because the test method name is `null`, so it appears to surefire as if this is
a separate method altogether that has never succeeded.
This is the problematic part of the code:
{code:java}
Map<String, List<TestMethodStats>> mergedTestHistoryResult = new HashMap<>();
// Merge all the stats for tests from listeners
for (TestSetRunListener listener : listeners) {
for (TestMethodStats methodStats : listener.getTestMethodStats()) {
List<TestMethodStats> currentMethodStats =
mergedTestHistoryResult.get(methodStats.getTestClassMethodName());
if (currentMethodStats == null) {
currentMethodStats = new ArrayList<>();
currentMethodStats.add(methodStats);
mergedTestHistoryResult.put(methodStats.getTestClassMethodName(),
currentMethodStats);
} else {
currentMethodStats.add(methodStats);
}
}
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
