[ https://issues.apache.org/jira/browse/SUREFIRE-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stefan Oehme updated SUREFIRE-1878: ----------------------------------- Description: The Surefire plugin currently covers two use cases with its `rerunFailingTestCount` option: 1. rerunFailingTestCount=0: The build will fail immediately, but I don't know if it was a real failure or a flake. This option is best for healthy code bases with very little or no flakiness. 2. rerunFailingTestCount>0: I will know when a test is flaky, but the build will be successful, so there is no pressure to fix the flakes. This helps when the situation is really bad and I just want to get some green build again to boost team morale :) I'd like to support a third use case: 3. rerunFailingTestCount>0 and a new option failOnFlake=true: The build will fail (so there is actually pressure to deal with flakiness) and I can tell the difference between real failures and flakes. I think this would be the best option for projects with a medium amount of flaky tests and a team that wants to take care of them. was: The Surefire plugin currently covers two use cases with its `rerunFailingTestCount` option: 1. rerunFailingTestCount=0: The build will fail on failed tests, but I don't know if it was a real failure or a flake. This option is best for healthy code bases with very little or no flakiness. 2. rerunFailingTestCount>0: I will know when a test is flaky, but the build will be successful, so there is no pressure to fix the flakes. This helps when the situation is really bad and I just want to get some green build again to boost team morale :) I'd like to support a third use case: 3. rerunFailingTestCount>0 and a new option failOnFlake=true: The build will fail (so there is actually pressure to deal with flakiness) and I can tell the difference between real failures and flakes. I think this would be the best option for projects with a medium amount of flaky tests and a team that wants to take care of them. > Add failOnFlake option > ---------------------- > > Key: SUREFIRE-1878 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1878 > Project: Maven Surefire > Issue Type: New Feature > Components: Maven Surefire Plugin > Reporter: Stefan Oehme > Priority: Minor > > The Surefire plugin currently covers two use cases with its > `rerunFailingTestCount` option: > 1. rerunFailingTestCount=0: The build will fail immediately, but I don't know > if it was a real failure or a flake. This option is best for healthy code > bases with very little or no flakiness. > 2. rerunFailingTestCount>0: I will know when a test is flaky, but the build > will be successful, so there is no pressure to fix the flakes. This helps > when the situation is really bad and I just want to get some green build > again to boost team morale :) > I'd like to support a third use case: > 3. rerunFailingTestCount>0 and a new option failOnFlake=true: The build will > fail (so there is actually pressure to deal with flakiness) and I can tell > the difference between real failures and flakes. I think this would be the > best option for projects with a medium amount of flaky tests and a team that > wants to take care of them. -- This message was sent by Atlassian Jira (v8.3.4#803005)