By design, acceptanceTest should be run with defaults (including ports). So if they aren’t run in containers they can’t be run in parallel.
Anthony > On Aug 21, 2018, at 11:15 AM, Kirk Lund <kl...@apache.org> wrote: > > Oops... I mean "orphaned processes" > > On Tue, Aug 21, 2018 at 11:13 AM, Kirk Lund <kl...@apache.org> wrote: > >> So, we need to disable running them in parallel and improve the tearDown >> to make sure these tests don't leave an orphaned tests behind. >> >> On Tue, Aug 21, 2018 at 11:11 AM, Sai Boorlagadda < >> sai.boorlaga...@gmail.com> wrote: >> >>> jdbc-connector acceptance tests need docker-in-docker (also >>> docker-compose) >>> to spin up mysql and postgres. >>> >>> On Tue, Aug 21, 2018 at 11:04 AM Dan Smith <dsm...@pivotal.io> wrote: >>> >>>> Actually, it looks like the problem is that we are *not* using docker >>>> containers for the acceptance tests. Check this out, in >>>> gradle/docker.gradle. Since acceptance tests use the default port, this >>>> means the test are guaranteed to be flaky, especially since we are >>> running >>>> them in parallel: >>>> >>>> // ACCEPTANCE TEST NEEDS DOCKER-COMPOSE TO WORK WITHIN DOCKER FIRST >>>> // acceptanceTest.configure(dockerConfig) >>>> >>>> I'm not sure what changed that is causing the tests to fail more often >>> now, >>>> but maybe a test ordering change? >>>> >>>> -Dan >>>> >>>> >>>> >>>> On Tue, Aug 21, 2018 at 10:52 AM, Kenneth Howe <kh...@pivotal.io> >>> wrote: >>>> >>>>> >>>>> >>>>>> On Aug 21, 2018, at 10:44 AM, Kirk Lund <kl...@apache.org> wrote: >>>>>> >>>>>> GEODE-5590 would seem to imply that GfshRule does not have an >>> adequate >>>>> safe >>>>>> guard? If it spawns a server process which binds to the default >>> server >>>>> port >>>>>> and that process persists after the test then we need better >>> tearDown. >>>>>> >>>>> Yes, that does appear to be the case. The current failures are >>> apparently >>>>> due to incomplete >>>>> teardown between tests within a test class. >>>>> >>>>> I am attempting to reproduce the failures on a consistent basis for >>>>> debugging the problem. >>>>> >>>>> >>>>>> Actually I thought we were using Docker to run each AcceptanceTest >>> in >>>>>> isolation. Then when the test finishes the Docker instances goes >>> away. >>>>> Did >>>>>> we stop using Docker for these? >>>>>> >>>>>> On Tue, Aug 21, 2018 at 10:25 AM, Sai Boorlagadda < >>>>> sai.boorlaga...@gmail.com >>>>>>> wrote: >>>>>> >>>>>>> DeployWithLargeJarTest & PutCommandWithJsonTest are flaky on >>> Develop. >>>>>>> >>>>>>> DeployWithLargeJarTest - >>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>>>>> develop/jobs/AcceptanceTest/builds/335 >>>>>>> PutCommandWithJsonTest - >>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>>>>> develop/jobs/AcceptanceTest/builds/334 >>>>>>> >>>>>>> On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda < >>>>>>> sai.boorlaga...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> The metrics job themselves will be green (as they complete to >>>> success) >>>>>>> but >>>>>>>> you can expand the get_metrics task output and see that build#20 >>>>> started >>>>>>>> reporting these failures, so probably these are due to recent >>> changes >>>>> on >>>>>>>> develop. I believe these metrics are from develop CI test runs. >>>>>>>> >>>>>>>> On Tue, Aug 21, 2018 at 10:15 AM Kirk Lund <kl...@apache.org> >>> wrote: >>>>>>>> >>>>>>>>> Those metrics show AcceptanceTests consistently GREEN. Do these >>>>> metrics >>>>>>>>> include test failures from pull request precheckin runs like >>> mine? >>>> Or >>>>>>> does >>>>>>>>> it just cover CI test runs? >>>>>>>>> >>>>>>>>> On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda < >>>>>>>>> sai.boorlaga...@gmail.com >>>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Metrics show these started failing recently. >>>>>>>>>> >>>>>>>>>> >>>>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>>>>> metrics/jobs/ >>>>>>>>>> GeodeAcceptanceTestMetrics/builds/20 >>>>>>>>>> >>>>>>>>>> On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund <kl...@apache.org> >>>> wrote: >>>>>>>>>> >>>>>>>>>>> Are PutCommandWithJsonTest and DeployWithLargeJarTest known to >>> be >>>>>>>>> flaky? >>>>>>>>>>> >>>>>>>>>>> My latest pull request failed with these two failures and all I >>>> did >>>>>>>>> was >>>>>>>>>>> extract LocalRegion.validateRegionName and improve unit >>> testing of >>>>>>>>>>> RegionNameValidation. No other tests failed for me. >>>>>>>>>>> >>>>>>>>>>>> Task :geode-assembly:acceptanceTest >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:619 >>>>> >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:620 >>>>> >>>>>>>>>>> >>>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>> PutCommandWithJsonTest >>>>>>>>>>>> putWithJsonString FAILED >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:621 >>>>> >>>>>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:622 >>>>> >>>>>>>>>>> at >>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>>>>>>> Method) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:623 >>>>> >>>>>>>>>>> at >>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>>>>>> NativeConstructorAccessorImpl.java:62) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:624 >>>>> >>>>>>>>>>> at >>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>>>>>>>>> DelegatingConstructorAccessorImpl.java:45) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:625 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. >>>>>>>>>> awaitIfNecessary(GfshScript.java:117) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:626 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >>>>>>>>>> GfshRule.java:135) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:627 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >>>>>>>>>> GfshScript.java:106) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:628 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>>>>> PutCommandWithJsonTest.putWithJsonString( >>>>> PutCommandWithJsonTest.java: >>>>>>> 55) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:629 >>>>> >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:630 >>>>> >>>>>>>>>>> >>>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>> DeployWithLargeJarTest >>>>>>>>>>>> deployLargeSetOfJars FAILED >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:631 >>>>> >>>>>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:632 >>>>> >>>>>>>>>>> at >>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>>>>>>> Method) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:633 >>>>> >>>>>>>>>>> at >>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>>>>>> NativeConstructorAccessorImpl.java:62) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:634 >>>>> >>>>>>>>>>> at >>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>>>>>>>>> DelegatingConstructorAccessorImpl.java:45) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:635 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. >>>>>>>>>> awaitIfNecessary(GfshScript.java:117) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:636 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >>>>>>>>>> GfshRule.java:135) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:637 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >>>>>>>>>> GfshScript.java:106) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:638 >>>>> >>>>>>>>>>> at >>>>>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>>>>> DeployWithLargeJarTest.deployLargeSetOfJars( >>>>>>> DeployWithLargeJarTest.java: >>>>>>>>>> 41) >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:639 >>>>> >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# >>> L5b60bc1a:640 >>>>> >>>>>>>>>>>> Task :geode-assembly:acceptanceTest FAILED >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >> >>
