[
https://issues.apache.org/jira/browse/SUREFIRE-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vlad Skarzhevskyy updated SUREFIRE-1760:
----------------------------------------
Affects Version/s: 2.22.1
2.22.2
3.0.0-M4
> Unbalanced number of test executions in forks (forkCount>1) results in no
> performance gain for jupiter/JUnit5 tests
> -------------------------------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1760
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1760
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support
> Affects Versions: 2.22.1, 2.22.2, 3.0.0-M4
> Reporter: Vlad Skarzhevskyy
> Priority: Major
>
> Running JUnit5 tests using maven-surefire-plugin with
> <reuseForks>true</reuseForks> e.g. mvn -DforkCount=8
> Results: number of tests in each fork is significantly unbalanced, *+some
> forks do not get tests at all+*..
> The end results is that on CI server the total test execution will take as
> long as longest fork and only 50% of the available resources are used.
> In our case we observed that our 3000 integration tests after migration to
> junit5 take 30 minutes insteads of 18 minutes as it was with junit 4.
> The balance is restored when running with reuseForks=false but this will
> cause additional overhead for complex integrations tests
> e.g. tests requiring additional setup , classpath scanning and so on..
> The results are similar with maven-surefire-plugin: 2.22.1, 2.22.2, 3.0.0-M4
> Maven 3.6.3 , openjdk version "1.8.0_242" on Linux
>
> BTW If junit-vintage-engine is used the same results observed for JUnit3 and
> JUnit4 tests.
>
> Consider example of 100 tests (10 classes 10 tests in each)
>
> [https://github.com/skarzhevskyy/range-junit/tree/master/range-junit5-surefire-fork]
>
> When running mvn -DforkCount=8 with extension that prints the number of
> tests executed in each fork getting following:
> {quote}Tests completed JVM Fork 8; total tests: 0; classes: 0
> Tests completed JVM Fork 4; total tests: 0; classes: 0
> Tests completed JVM Fork 1; total tests: 0; classes: 0
> Tests completed JVM Fork 6; total tests: 0; classes: 0
> Tests completed JVM Fork 3; total tests: 0; classes: 0
> Tests completed JVM Fork 2; total tests: 0; classes: 0
> Tests completed JVM Fork 7; total tests: 30; classes: 3
> Tests completed JVM Fork 5; total tests: 70; classes: 7
> {quote}
>
> mvn -DforkCount=4
> {quote}Tests completed JVM Fork 2; total tests: 0; classes: 0
> Tests completed JVM Fork 3; total tests: 10; classes: 1
> Tests completed JVM Fork 4; total tests: 40; classes: 4
> Tests completed JVM Fork 1; total tests: 50; classes: 5
> {quote}
>
> The distribution is also random, results may vary between maven runs:
> mvn -DforkCount=4
> {quote}Tests completed JVM Fork 4; total tests: 0; classes: 0
> Tests completed JVM Fork 2; total tests: 10; classes: 1
> Tests completed JVM Fork 1; total tests: 30; classes: 3
> Tests completed JVM Fork 3; total tests: 60; classes: 6
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
