Ramanan Rajendran created SUREFIRE-2087:
-------------------------------------------
Summary: rerunFailingTestsCount incorrectly marks failed
parametrized test as pass
Key: SUREFIRE-2087
URL: https://issues.apache.org/jira/browse/SUREFIRE-2087
Project: Maven Surefire
Issue Type: Bug
Reporter: Ramanan Rajendran
rerunFailingTestsCount is not working as expected with junit5
`@ParameterizedTest`
If 1st iteration of the test passes and the 2nd one fails, it still marks the
test as flaky passed.
Eg: In the following run, iteration 1 with input A passed but iteration 2 with
input B failed. Iteration 2 was rerun 2 times and failed in all the attempts.
Instead of failing TestA, it's marked as flaky giving an overall pass for the
test suite. I believe it's using the Run 1 PASS as a gate to mark the whole
test as a pass.
[INFO] Run 1: PASS
[ERROR] Run 2: TestA.iteration2:101 1 expectation failed.
Expected status code <400> doesn't match actual status code <401>.
[ERROR] Run 3: TestA.iteration2:101 1 expectation failed.
Expected status code <400> doesn't match actual status code <401>.
[ERROR] Run 4: TestA.iteration2:101 1 expectation failed.
<400> doesn't match actual status code <401>.
Tests run: 23, Failures: 0, Errors: 0, Skipped: 2, Flakes: 1
--
This message was sent by Atlassian Jira
(v8.20.7#820007)