[
https://issues.apache.org/jira/browse/SUREFIRE-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Oehme updated SUREFIRE-1878:
-----------------------------------
Description:
The Surefire plugin currently covers two use cases with its
`rerunFailingTestCount` option:
1. rerunFailingTestCount=0: The build will fail immediately, but I don't know
if it was a real failure or a flake. This option is best for healthy code bases
with very little or no flakiness.
2. rerunFailingTestCount>0: I will know when a test is flaky, but the build
will be successful, so there is no pressure to fix the flakes. This helps when
the situation is really bad and I just want to get some green build again to
boost team morale :)
I'd like to support a third use case:
3. rerunFailingTestCount>0 and a new option failOnFlake=true: The build will
fail (so there is actually pressure to deal with flakiness) and I can tell the
difference between real failures and flakes. I think this would be the best
option for projects with a medium amount of flaky tests and a team that wants
to take care of them.
was:
The Surefire plugin currently covers two use cases with its
`rerunFailingTestCount` option:
1. rerunFailingTestCount=0: The build will fail on failed tests, but I don't
know if it was a real failure or a flake. This option is best for healthy code
bases with very little or no flakiness.
2. rerunFailingTestCount>0: I will know when a test is flaky, but the build
will be successful, so there is no pressure to fix the flakes. This helps when
the situation is really bad and I just want to get some green build again to
boost team morale :)
I'd like to support a third use case:
3. rerunFailingTestCount>0 and a new option failOnFlake=true: The build will
fail (so there is actually pressure to deal with flakiness) and I can tell the
difference between real failures and flakes. I think this would be the best
option for projects with a medium amount of flaky tests and a team that wants
to take care of them.
> Add failOnFlake option
> ----------------------
>
> Key: SUREFIRE-1878
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1878
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Maven Surefire Plugin
> Reporter: Stefan Oehme
> Priority: Minor
>
> The Surefire plugin currently covers two use cases with its
> `rerunFailingTestCount` option:
> 1. rerunFailingTestCount=0: The build will fail immediately, but I don't know
> if it was a real failure or a flake. This option is best for healthy code
> bases with very little or no flakiness.
> 2. rerunFailingTestCount>0: I will know when a test is flaky, but the build
> will be successful, so there is no pressure to fix the flakes. This helps
> when the situation is really bad and I just want to get some green build
> again to boost team morale :)
> I'd like to support a third use case:
> 3. rerunFailingTestCount>0 and a new option failOnFlake=true: The build will
> fail (so there is actually pressure to deal with flakiness) and I can tell
> the difference between real failures and flakes. I think this would be the
> best option for projects with a medium amount of flaky tests and a team that
> wants to take care of them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
