[
http://jira.codehaus.org/browse/SUREFIRE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Fabulich closed SUREFIRE-476.
---------------------------------
Resolution: Won't Fix
I don't think this patch is logically correct. Logically, a test can have
three results: pass, fail or skip. Unfortunately, most people assume that
there are only two possible results (pass or fail); they furthermore assume
that "fail" means the same thing as "did not pass."
I don't think we want an option to decide whether "skip" means "fail" or
whether "skip" means "pass." Skip is separate; it's not a success, and it's
not a failure.
If a test is skipped, then it is not a success. That doesn't mean it's a
failure, but it also means that you don't have 100% success. Perhaps what
you'd want is a separate "failure percentage." If all tests are skipped, the
"pass" percentage is 0 and the failure percentage is 0.
Regardless, I'm pretty sure it's wrong to make the "successful percentage" be
100% when there are skipped tests, because those tests did not succeed.
> Skipped test should not be considered as "failed test"
> ------------------------------------------------------
>
> Key: SUREFIRE-476
> URL: http://jira.codehaus.org/browse/SUREFIRE-476
> Project: Maven Surefire
> Issue Type: Improvement
> Components: report plugin
> Affects Versions: 2.4.2
> Reporter: Baptiste MATHUS
> Attachments: surefire-report-skippedIsSuccess.patch
>
>
> I guess the plugin should allow people to decide whether skipped test is the
> same as a failed test, as it is at the moment in the report plugin.
> As you know, with the surefire plugin (not the report one), a skipped test is
> not considered as a failed one, as it doesn't make the whole test phase fail,
> isn't it? So, I guess there should be at least an option to choose how to
> treat skipped tests when reporting.
> As a small modification, I'm attaching a small patch to not lower the
> "success rate" (for example, with two tests one successful, one skipped, the
> success rate stays 100%). This patch uses an option -DskippedIsFailed=false
> that lets the current behaviour be if you don't use this switch (the testcase
> was also updated to test this new switch).
> I think there could be a good idea to introduce a specific part in the report
> showing the skipped tests. What do you think? If you agree, I could try and
> propose a patch for it.
> Cheers.
> PS: the patch was applied against the 2.4.2 tag. If you want me to update it
> against another source version, please let me know.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
