Qingzhou Luo created SUREFIRE-1087:
--------------------------------------
Summary: New option rerunFailingTestsCount
Key: SUREFIRE-1087
URL: https://jira.codehaus.org/browse/SUREFIRE-1087
Project: Maven Surefire
Issue Type: New Feature
Reporter: Qingzhou Luo
Pull Request 40:
https://github.com/apache/maven-surefire/pull/40
Add rerunFailingTestsCount option for maven surefire to rerun failing tests
immediately after they fail.
When rerunFailingTestsCount is set to a value k larger than 0, a failing
test will get re-run
up to k times until it passes. If a test passes in any of its reruns,
the build will be marked as successful and the test will count as a
flake (or flaky test). If it fails all those k times then it will
still be marked as a failed test.
In the console output all the flaky tests will be count as "Flakes:
N". The generated test report XML file is augmented with additional
information, while still being compatible with existing consumers
(such as Jenkins). A flaky test will have <flakyFailure> or/and
<flakyError> under its <testcase> element, to store all the flaky
runs' information (such as output, stackTrace). So existing consumers
will still consider it as a passing test, while potential future
consumers can parse those flaky runs information. A failing test will
still have <failure> or <error> under <testcase>, but all the
subsequent re-run information will be stored under <rerunFailure> or
<rerunError>. So existing consumers will still be able to see it's a
failed test and parse its failure information, and potential future
consumers will be able to get all the flaky runs.
It is implemented by keeping a map between test full class name and a
map between all its test methods and the list of runs. It also takes
into account Fork and Parallel and have them covered by integration
tests.
Currently only supports JUnit4.x
--
This message was sent by Atlassian JIRA
(v6.1.6#6162)
