Ken and I are paired up to containerize now.
> On Aug 21, 2018, at 2:20 PM, Dan Smith <dsm...@pivotal.io> wrote: > > I see we have a few JIRAs that were filed related to this issue. I think I > cleaned them up, so whoever is working on fixing you can use this JIRA - > https://issues.apache.org/jira/browse/GEODE-5601. > > Until this is fixed, let's not create new JIRAs for AcceptanceTest failures. > > -Dan > >> On Tue, Aug 21, 2018 at 1:02 PM, Jacob Barrett <jbarr...@pivotal.io> wrote: >> >> Until docker on docker is supported for acceptance tests you can disable >> the parallelism on forks with -DdunitParallelForks=1 when running >> acceptanceTest. We can do the same in the CI for now too. :( >> >> The change for the CI can be found in >> ci/pipelines/shared/variablesomething.yml. >> >> -Jake >> >> >>> On Tue, Aug 21, 2018 at 11:04 AM Dan Smith <dsm...@pivotal.io> wrote: >>> >>> Actually, it looks like the problem is that we are *not* using docker >>> containers for the acceptance tests. Check this out, in >>> gradle/docker.gradle. Since acceptance tests use the default port, this >>> means the test are guaranteed to be flaky, especially since we are >> running >>> them in parallel: >>> >>> // ACCEPTANCE TEST NEEDS DOCKER-COMPOSE TO WORK WITHIN DOCKER FIRST >>> // acceptanceTest.configure(dockerConfig) >>> >>> I'm not sure what changed that is causing the tests to fail more often >> now, >>> but maybe a test ordering change? >>> >>> -Dan >>> >>> >>> >>>> On Tue, Aug 21, 2018 at 10:52 AM, Kenneth Howe <kh...@pivotal.io> wrote: >>>> >>>> >>>> >>>>> On Aug 21, 2018, at 10:44 AM, Kirk Lund <kl...@apache.org> wrote: >>>>> >>>>> GEODE-5590 would seem to imply that GfshRule does not have an >> adequate >>>> safe >>>>> guard? If it spawns a server process which binds to the default >> server >>>> port >>>>> and that process persists after the test then we need better >> tearDown. >>>>> >>>> Yes, that does appear to be the case. The current failures are >> apparently >>>> due to incomplete >>>> teardown between tests within a test class. >>>> >>>> I am attempting to reproduce the failures on a consistent basis for >>>> debugging the problem. >>>> >>>> >>>>> Actually I thought we were using Docker to run each AcceptanceTest in >>>>> isolation. Then when the test finishes the Docker instances goes >> away. >>>> Did >>>>> we stop using Docker for these? >>>>> >>>>> On Tue, Aug 21, 2018 at 10:25 AM, Sai Boorlagadda < >>>> sai.boorlaga...@gmail.com >>>>>> wrote: >>>>> >>>>>> DeployWithLargeJarTest & PutCommandWithJsonTest are flaky on >> Develop. >>>>>> >>>>>> DeployWithLargeJarTest - >>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>>>> develop/jobs/AcceptanceTest/builds/335 >>>>>> PutCommandWithJsonTest - >>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>>>> develop/jobs/AcceptanceTest/builds/334 >>>>>> >>>>>> On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda < >>>>>> sai.boorlaga...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> The metrics job themselves will be green (as they complete to >>> success) >>>>>> but >>>>>>> you can expand the get_metrics task output and see that build#20 >>>> started >>>>>>> reporting these failures, so probably these are due to recent >> changes >>>> on >>>>>>> develop. I believe these metrics are from develop CI test runs. >>>>>>> >>>>>>> On Tue, Aug 21, 2018 at 10:15 AM Kirk Lund <kl...@apache.org> >> wrote: >>>>>>> >>>>>>>> Those metrics show AcceptanceTests consistently GREEN. Do these >>>> metrics >>>>>>>> include test failures from pull request precheckin runs like mine? >>> Or >>>>>> does >>>>>>>> it just cover CI test runs? >>>>>>>> >>>>>>>> On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda < >>>>>>>> sai.boorlaga...@gmail.com >>>>>>>>> wrote: >>>>>>>> >>>>>>>>> Metrics show these started failing recently. >>>>>>>>> >>>>>>>>> >>>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>>>> metrics/jobs/ >>>>>>>>> GeodeAcceptanceTestMetrics/builds/20 >>>>>>>>> >>>>>>>>> On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund <kl...@apache.org> >>> wrote: >>>>>>>>> >>>>>>>>>> Are PutCommandWithJsonTest and DeployWithLargeJarTest known to >> be >>>>>>>> flaky? >>>>>>>>>> >>>>>>>>>> My latest pull request failed with these two failures and all I >>> did >>>>>>>> was >>>>>>>>>> extract LocalRegion.validateRegionName and improve unit testing >> of >>>>>>>>>> RegionNameValidation. No other tests failed for me. >>>>>>>>>> >>>>>>>>>>> Task :geode-assembly:acceptanceTest >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:619 >>>> >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:620 >>>> >>>>>>>>>> >>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>> PutCommandWithJsonTest >>>>>>>>>>> putWithJsonString FAILED >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:621 >>>> >>>>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:622 >>>> >>>>>>>>>> at >>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>>>>>> Method) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:623 >>>> >>>>>>>>>> at >>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>>>>> NativeConstructorAccessorImpl.java:62) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:624 >>>> >>>>>>>>>> at >>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>>>>>>>> DelegatingConstructorAccessorImpl.java:45) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:625 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. >>>>>>>>> awaitIfNecessary(GfshScript.java:117) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:626 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >>>>>>>>> GfshRule.java:135) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:627 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >>>>>>>>> GfshScript.java:106) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:628 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>>>> PutCommandWithJsonTest.putWithJsonString( >>>> PutCommandWithJsonTest.java: >>>>>> 55) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:629 >>>> >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:630 >>>> >>>>>>>>>> >>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>> DeployWithLargeJarTest >>>>>>>>>>> deployLargeSetOfJars FAILED >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:631 >>>> >>>>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:632 >>>> >>>>>>>>>> at >>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>>>>>> Method) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:633 >>>> >>>>>>>>>> at >>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>>>>> NativeConstructorAccessorImpl.java:62) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:634 >>>> >>>>>>>>>> at >>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>>>>>>>> DelegatingConstructorAccessorImpl.java:45) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:635 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. >>>>>>>>> awaitIfNecessary(GfshScript.java:117) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:636 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >>>>>>>>> GfshRule.java:135) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:637 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >>>>>>>>> GfshScript.java:106) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:638 >>>> >>>>>>>>>> at >>>>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>>>> DeployWithLargeJarTest.deployLargeSetOfJars( >>>>>> DeployWithLargeJarTest.java: >>>>>>>>> 41) >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:639 >>>> >>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/ >> 19680#L5b60bc1a:640 >>>> >>>>>>>>>>> Task :geode-assembly:acceptanceTest FAILED >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>>> >>> >>
