Semyon Danilov created SUREFIRE-2039:
----------------------------------------
Summary: Skipped test classes are getting into the non-skipped
test classes reports
Key: SUREFIRE-2039
URL: https://issues.apache.org/jira/browse/SUREFIRE-2039
Project: Maven Surefire
Issue Type: Bug
Affects Versions: 3.0.0-M5
Reporter: Semyon Danilov
Consider a simple scenario, two test classes, `ATest` and `BTest`, one
Disabled and one not respectively, each has three test methods. After running
`mvn surefire:test` the report will indicate that 4 tests were run and one of
them was skipped:
{noformat}
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.example.test.BTest
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.062
s - in org.example.test.BTest
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1
{noformat}
And if we enable ATest and disable BTest, then report will only mention running
three tests, none skipped:
{noformat}
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.example.test.ATest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 s
- in org.example.test.ATest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
{noformat}
This behavior is the result of the way how
`org.apache.maven.plugin.surefire.report.TestSetRunListener` handles skipped
test. It adds them to the `detailsForThis` and only print after test set is
finished which happens when a non-disabled test set is finished thus adding
skipped test results to the statistics of a non-skipped test set. And, of
course, if the disabled test is ran after the non-disabled test, it won't be
reported at all.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
