[
https://jira.codehaus.org/browse/SUREFIRE-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=354529#comment-354529
]
Tibor Digana commented on SUREFIRE-779:
---------------------------------------
@Kristian
@Andreas
I am not sure how this should be fixed, and whether to fix it.
The behavior described by @Lieven presents in 2.18-SNAPSHOT.
I have a problem with calculating the number of passed tests
(Tests run - Failures - Errors).
If this was fixed, then the #passed tests would be 1 - 2 = -1.
Tests run: 1, Failures: 2, Errors: 0, Skipped: 0
How is this handled by e.g. Jenkins?
Do the build systems rely on this summary?
> Surefire reports wrong number of failed tests when using JUnit's
> ErrorCollector rule
> ------------------------------------------------------------------------------------
>
> Key: SUREFIRE-779
> URL: https://jira.codehaus.org/browse/SUREFIRE-779
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
> Affects Versions: 2.10
> Reporter: Lieven Doclo
> Assignee: Tibor Digana
>
> When running a test that contains an ErrorCollector, test failures are not
> correct when an errorcollector contains failures. For example:
> {noformat}
> public class ExampleTest {
> @Rule
> public ErrorCollector errorCollector = new ErrorCollector();
> @Test
> public void testWithErrorCollector() {
> errorCollector.checkThat(true, is(false));
> errorCollector.checkThat(3, is(4));
> }
> }
> {noformat}
> Surefire will reports 2 tests run (of which all failed). However, when you
> fix the same test so that no checks fail, Surefire will report 1 test run.
> This is incorrect, as this will result in false information in systems like
> Hudson or Sonar, the number of testcase should not fluctuate. Surefire should
> only report 1 test run, as that reflects the actual situation.
--
This message was sent by Atlassian JIRA
(v6.1.6#6162)
