Debraj Manna created SUREFIRE-2195:
--------------------------------------
Summary: Surefire reports are not correct when running tests in
parallel using JUnit5
Key: SUREFIRE-2195
URL: https://issues.apache.org/jira/browse/SUREFIRE-2195
Project: Maven Surefire
Issue Type: Bug
Components: JUnit 5.x support
Affects Versions: 3.1.2, 3.0.0-M7
Reporter: Debraj Manna
Attachments: image.png
The issue has been discussed in the [Maven mailing
list|https://lists.apache.org/thread/6k10kmxzz75rz8mqn3pl1ltvfltw87m9]. I am
creating a bug here as suggested in the mailing list
Cross-posting it here for completeness
My {{junit-platform.properties}} looks like below
{code:java}
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.config.strategy=fixed
junit.jupiter.execution.parallel.config.fixed.parallelism=8
junit.jupiter.execution.parallel.mode.default=same_thread
junit.jupiter.execution.parallel.mode.classes.default=concurrent
{code}
I am observing the surefire reports that are getting generated are not correct
I have a test class named
{{com.spotnana.servicetests.backoffice.invoicing.AirInvoiceServiceTest}}
contains just a single test, {{testAirTicketing()}}.
But in the report file,
TEST-com.spotnana.servicetests.backoffice.invoicing.AirInvoiceServiceTest.xml.
I see like below
{code:xml}
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd";
version="3.0"
name="com.spotnana.servicetests.backoffice.invoicing.AirInvoiceServiceTest"
time="44.711" tests="2" errors="1" skipped="0" failures="0”>
…
<testcase name="testCrossPnrRefund"
classname="com.spotnana.servicetests.analytics.ingestion.pnr.AirServiceTest"
time="5.789"/>
<testcase name="testAirTicketing"
classname="com.spotnana.servicetests.backoffice.invoicing.AirInvoiceServiceTest"
time="33.331”>
{code}
It seems to be picking tests from some other class like {{AirServiceTest}} as
in this case.
The console logs are also not correct. Please check the attached screenshot. In
the attached screenshot of the console logs, it is saying {{Test Run 5}} but
the {{AirInvoiceServiceTest}} contains only 1 test. And also the report shows
random data. For example, sometimes it says {{Test Run 5}} , sometimes it says
{{Test Run 2}}, etc.
I am observing it as part of a big project consistently with every run. I am
trying to reproduce it in a small example. If I am able to do I will share it.
In the mean time if some debug logs are needed, let me know I will be happy to
share them.
Junit5 Version - 5.9.3
I am observing the issue with Junit5 Version 5.10.0 also. I tried reducing the
junit.jupiter.execution.parallel.config.fixed.parallelism to 3 but there also I
am getting the same problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
