[
https://issues.apache.org/jira/browse/SUREFIRE-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17940092#comment-17940092
]
Pavlo Shevchenko commented on SUREFIRE-2298:
--------------------------------------------
This change also breaks the reporting of Cucumber tests.
Steps to reproduce:
# Set up the sample Cucumber project using "cucumber-archetype"
([ref|https://cucumber.io/docs/guides/10-minute-tutorial/#create-an-empty-cucumber-project]):
# For Surefire 3.5.2, the console output will be:
{code:java}
[INFO] Running hellocucumber.RunCucumberTestScenario: The example
# hellocucumber/example.feature:3
Given an example scenario #
hellocucumber.StepDefinitions.anExampleScenario()
When all step definitions are implemented #
hellocucumber.StepDefinitions.allStepDefinitionsAreImplemented()
Then the scenario passes #
hellocucumber.StepDefinitions.theScenarioPasses()
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.065 s
-- in hellocucumber.RunCucumberTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 {code}
, and the XML report ("TEST-hellocucumber.RunCucumberTest.xml") will contain a
single test case:
{code:java}
<testcase name="An example - The example" classname="An example" time="0.028">
{code}
# With Surefire 3.5.3, on the other hand, the output will say that no tests
were run:
{code:java}
[INFO] Running hellocucumber.RunCucumberTestScenario: The example
# hellocucumber/example.feature:3
Given an example scenario #
hellocucumber.StepDefinitions.anExampleScenario()
When all step definitions are implemented #
hellocucumber.StepDefinitions.allStepDefinitionsAreImplemented()
Then the scenario passes #
hellocucumber.StepDefinitions.theScenarioPasses()
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 s
-- in hellocucumber.RunCucumberTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 {code}
, and the XML report will contain no test cases.
> @Nested test classes are not reported properly
> ----------------------------------------------
>
> Key: SUREFIRE-2298
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2298
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 3.5.3
> Reporter: Leonard Brünings
> Priority: Major
>
> Given this example test you can see that the {{@Nested}} test classes are not
> reported correctly. In 3.5.2 and earlier all executions land in the
> {{TEST-BaseNestedTest$Inner.xml}} and with the changes in 3.5.3 they are
> distributed in all files. However, the nested classes belong to the enclosing
> class and the reporting should reflect that, so it would be more accurate to
> report the nested executions as part of the enclosing tests (see below).
> [~marcphilipp] please chime in.
>
> {code:java}
> import org.junit.jupiter.api.*;
> abstract class BaseNestedTest {
> @Test
> void outerTest() {
> }
>
> @Nested
> class Inner {
> @Test
> void innerTest() {
> }
> }
> }
> class FirstNestedTest extends BaseNestedTest {
> }
> class SecondNestedTest extends BaseNestedTest {
> }{code}
> This was most likely caused by
> [https://github.com/apache/maven-surefire/commit/5aeca19716dedc4cbcab8d68dfc74a99f50ce9ee]
> 3.5.2
> {code:java|title=TEST-FirstNestedTest.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="FirstNestedTest" time="0.019" tests="0" errors="0"
> skipped="0" failures="0">
> </testsuite>
> {code}
> {code:java|title=TEST-SecondNestedTest.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="SecondNestedTest" time="0.004" tests="0" errors="0"
> skipped="0" failures="0">
> </testsuite>
> {code}
> {code:java|title=TEST-BaseNestedTest$Inner.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="BaseNestedTest$Inner" time="0.001" tests="2"
> errors="0" skipped="0" failures="0">
> <testcase name="outerTest" classname="FirstNestedTest" time="0.007"/>
> <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
> <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
> <testcase name="outerTest" classname="SecondNestedTest" time="0.001"/>
> </testsuite>
> {code}
> -------------------
> 3.5.3
> {code:java|title=TEST-FirstNestedTest.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="FirstNestedTest" time="0.019" tests="1" errors="0"
> skipped="0" failures="0">
> <testcase name="outerTest" classname="FirstNestedTest" time="0.007"/>
> </testsuite>
> {code}
> {code:java|title=TEST-SecondNestedTest.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="SecondNestedTest" time="0.003" tests="1" errors="0"
> skipped="0" failures="0">
> </properties>
> <testcase name="outerTest" classname="SecondNestedTest" time="0.001"/>
> </testsuite>
> {code}
> {code:java|title=TEST-BaseNestedTest$Inner.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="BaseNestedTest$Inner" time="0.001" tests="1"
> errors="0" skipped="0" failures="0">
> <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.001"/>
> <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
> </testsuite>
> {code}
>
> -------------------
> Expected reporting
>
> {code:java|title=TEST-FirstNestedTest.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="FirstNestedTest" time="0.019" tests="1" errors="0"
> skipped="0" failures="0">
> <testcase name="outerTest" classname="FirstNestedTest" time="0.007"/>
> <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.001"/>
> </testsuite>
> {code}
> {code:java|title=TEST-SecondNestedTest.xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd";
> version="3.0.2" name="SecondNestedTest" time="0.003" tests="1" errors="0"
> skipped="0" failures="0">
> </properties>
> <testcase name="outerTest" classname="SecondNestedTest" time="0.001"/>
> <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
> </testsuite>
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
