[ https://issues.apache.org/jira/browse/SUREFIRE-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631284#comment-14631284 ]
Tibor Digana commented on SUREFIRE-1169: ---------------------------------------- Basically you spread out the tests randomly across the forks but this solution does not have load balancer. The streamer can average the load across the forks. > JUnit / Arquillian lifecycle friendly test execution with forkCount > 1 and > reuseForks = true > --------------------------------------------------------------------------------------------- > > Key: SUREFIRE-1169 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1169 > Project: Maven Surefire > Issue Type: Improvement > Components: process forking > Affects Versions: 2.18.1 > Reporter: Falko Modler > Attachments: SUREFIRE-1169_forksExecuteTestsOneByOne.patch > > > The current approach to "stream" each test class name to a fork via > {{TestProvidingInputStream}} yields a good "load balancing" accross the forks > but it *triggers the entire test lifecycle for each test*. > With {{forkCount = 1}}, all tests are executed in one set but with > {{forkCount = n}} (n > 1) each test is a separate "set" (so to say). > This is very problematic in case you or a test framwork you are using relies > on a certain lifecycle. > [Arquillian|http://arquillian.org/] for example ties various "events" to > JUnit's lifecycle methods, like {{AfterSuite}} to > {{RunListener.testRunFinished(...)}} which triggers the shutdown of the > managed server etc. > When using multiple forks, {{RunListener.testRunFinished(...)}} is called for > *every* single test class, triggering {{AfterSuite}} for every single test, > although the fork will receive further tests after that which should "reuse" > the server. > This is just an example. In fact the whole JUnit / Arquillian lifecycle is > inconsistent when using multiple forks. > From a user perspective I wouldn't expect this behaviour: > As {{forkCount = 1}} (and {{reusableForks = true}}) triggers > {{RunListener.testRunFinished(...)}} *once*, {{forkCount = n}} (and > {{reusableForks = true}}) should trigger that method *n* times, not *x* times. > To be fair, the > [documentation|https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html] > *does* contain a pointer to that problem by saying: > {quote} > When using reuseForks=true and a forkCount value larger than one, test > classes are handed over to the forked process one-by-one. > {quote} > But the consequences remain very unclear. > *(Possible) Solution:* > I took a stab at this and implemented an "eager test distribution" to the > forks in {{ForkStart.java}} and disabled streaming. Please see attached patch > (to be applied against project root, 2.18.1). > Patch Details: > - New config property: {{ForkConfiguration.forksExecuteTestsOneByOne}}, set > via Mojo (default is true for backward compatibility, name is debatable) > - When {{forksExecuteTestsOneByOne}} ist set to false, the {{messageQueue}} > in {{ForkStarter.runSuitesForkOnceMultiple(...)}} is *not* wrapped in fork > specific {{TestProvidingInputStream}} instances to be consumed bit by bit > later on. > Instead, the queue is consumed directly and each test class name is assigned > to the respective fork by creating a copy of the {{providerProperties}} which > is filled individually for each fork. > E.g. for three forks and eight tests: > -- fork 1 executes test 1, 4 and 7 > -- fork 2 executes test 2, 5 and 8 > -- fork 3 executes test 3 and 6 > - To have a clean {{providerProperties}} template I had to move > {{DefaultScanResult.writeTo(...)}} to the respective private methods. > Otherwise the properties would have contained *all* tests already. > - I refactored some methods in {{ForkStarter}} to enhance readability and to > reduce code duplication. > - The patch does *not* contain a test for the new behaviour but all existing > tests passed. > {{forksExecuteTestsOneByOne = false}} now leads to a consistent lifecycle. > This solution has one downside: One or more forks could be overloaded while > other forks could "underloaded" because you cannot say how long each test > will take. runOrder=balanced could help here but has yet to be implemented > for forks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)