A good rule of thumb is to never test with default ports unless the test runs in a container. Otherwise, running tests in parallel will fail miserably.
On Tue, Aug 21, 2018 at 10:44 AM, Kirk Lund <kl...@apache.org> wrote: > GEODE-5590 would seem to imply that GfshRule does not have an adequate > safe guard? If it spawns a server process which binds to the default server > port and that process persists after the test then we need better tearDown. > > Actually I thought we were using Docker to run each AcceptanceTest in > isolation. Then when the test finishes the Docker instances goes away. Did > we stop using Docker for these? > > On Tue, Aug 21, 2018 at 10:25 AM, Sai Boorlagadda < > sai.boorlaga...@gmail.com> wrote: > >> DeployWithLargeJarTest & PutCommandWithJsonTest are flaky on Develop. >> >> DeployWithLargeJarTest - >> https://concourse.apachegeode-ci.info/teams/main/pipelines/d >> evelop/jobs/AcceptanceTest/builds/335 >> PutCommandWithJsonTest - >> https://concourse.apachegeode-ci.info/teams/main/pipelines/d >> evelop/jobs/AcceptanceTest/builds/334 >> >> On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda < >> sai.boorlaga...@gmail.com> >> wrote: >> >> > The metrics job themselves will be green (as they complete to success) >> but >> > you can expand the get_metrics task output and see that build#20 started >> > reporting these failures, so probably these are due to recent changes on >> > develop. I believe these metrics are from develop CI test runs. >> > >> > On Tue, Aug 21, 2018 at 10:15 AM Kirk Lund <kl...@apache.org> wrote: >> > >> >> Those metrics show AcceptanceTests consistently GREEN. Do these metrics >> >> include test failures from pull request precheckin runs like mine? Or >> does >> >> it just cover CI test runs? >> >> >> >> On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda < >> >> sai.boorlaga...@gmail.com >> >> > wrote: >> >> >> >> > Metrics show these started failing recently. >> >> > >> >> > >> >> https://concourse.apachegeode-ci.info/teams/main/pipelines/m >> etrics/jobs/ >> >> > GeodeAcceptanceTestMetrics/builds/20 >> >> > >> >> > On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund <kl...@apache.org> wrote: >> >> > >> >> > > Are PutCommandWithJsonTest and DeployWithLargeJarTest known to be >> >> flaky? >> >> > > >> >> > > My latest pull request failed with these two failures and all I did >> >> was >> >> > > extract LocalRegion.validateRegionName and improve unit testing of >> >> > > RegionNameValidation. No other tests failed for me. >> >> > > >> >> > > > Task :geode-assembly:acceptanceTest >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:619 >> > >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:620 >> > >> >> > > >> >> org.apache.geode.management.internal.cli.commands.PutCommand >> WithJsonTest >> >> > > > putWithJsonString FAILED >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:621 >> > >> >> > > org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:622 >> > >> >> > > at >> >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> >> > > Method) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:623 >> > >> >> > > at >> >> > > sun.reflect.NativeConstructorAccessorImpl.newInstance( >> >> > NativeConstructorAccessorImpl.java:62) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:624 >> > >> >> > > at >> >> > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> >> > DelegatingConstructorAccessorImpl.java:45) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:625 >> > >> >> > > at >> >> > > org.apache.geode.test.junit.rules.gfsh.GfshScript. >> >> > awaitIfNecessary(GfshScript.java:117) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:626 >> > >> >> > > at >> >> > > org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >> >> > GfshRule.java:135) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:627 >> > >> >> > > at >> >> > > org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >> >> > GfshScript.java:106) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:628 >> > >> >> > > at >> >> > > org.apache.geode.management.internal.cli.commands. >> >> > PutCommandWithJsonTest.putWithJsonString(PutCommandWithJsonT >> est.java:55) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:629 >> > >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:630 >> > >> >> > > >> >> org.apache.geode.management.internal.cli.commands.DeployWith >> LargeJarTest >> >> > > > deployLargeSetOfJars FAILED >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:631 >> > >> >> > > org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:632 >> > >> >> > > at >> >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> >> > > Method) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:633 >> > >> >> > > at >> >> > > sun.reflect.NativeConstructorAccessorImpl.newInstance( >> >> > NativeConstructorAccessorImpl.java:62) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:634 >> > >> >> > > at >> >> > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> >> > DelegatingConstructorAccessorImpl.java:45) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:635 >> > >> >> > > at >> >> > > org.apache.geode.test.junit.rules.gfsh.GfshScript. >> >> > awaitIfNecessary(GfshScript.java:117) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:636 >> > >> >> > > at >> >> > > org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >> >> > GfshRule.java:135) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:637 >> > >> >> > > at >> >> > > org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >> >> > GfshScript.java:106) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:638 >> > >> >> > > at >> >> > > org.apache.geode.management.internal.cli.commands. >> >> > DeployWithLargeJarTest.deployLargeSetOfJars(DeployWithLargeJ >> arTest.java: >> >> > 41) >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:639 >> > >> >> > > <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:640 >> > >> >> > > > Task :geode-assembly:acceptanceTest FAILED >> >> > > >> >> > >> >> >> > >> > >
