[
https://issues.apache.org/jira/browse/SUREFIRE-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elliotte Rusty Harold updated SUREFIRE-1219:
--------------------------------------------
Priority: Minor (was: Major)
> skipAfterFailureCount should not count flaky tests as failures when using
> rerunFailingTestsCount
> ------------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1219
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1219
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.19.1
> Reporter: Sean Flanigan
> Assignee: Tibor Digana
> Priority: Minor
> Attachments: SUREFIRE-1219-tests.zip
>
>
> According to
> https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html
> "failed tests within re-run phase are not included in
> skipAfterFailureCount". For instance, if skipAfterFailureCount is 1, but
> every test passes on the second attempt, the tests are "flaky" but not
> "failures". This should be reflected in summaries, in XML result files and
> in fail fast (skipAfterFailureCount), but this is not the case.
> If I have 10 flaky-but-not-failing tests, set rerunFailingTestsCount=1 and
> skipAfterFailureCount=5:
> 1. I would expect something like this to be printed on the console:
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Flakes: 10
> 2. The XML report should list <flakyFailure>s but no <failures>.
> 3. All tests should be run, not just 5 of them, in other words
> skipAfterFailureCount should not count the flaky tests as part of the failure
> count.
> I have some tests demonstrating the problem, which I will attach to this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
