Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Jacob Barrett
Ken and I are paired up to containerize now. > On Aug 21, 2018, at 2:20 PM, Dan Smith wrote: > > I see we have a few JIRAs that were filed related to this issue. I think I > cleaned them up, so whoever is working on fixing you can use this JIRA - > https://issues.apache.org/jira/browse/GEODE-560

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Dan Smith
I see we have a few JIRAs that were filed related to this issue. I think I cleaned them up, so whoever is working on fixing you can use this JIRA - https://issues.apache.org/jira/browse/GEODE-5601. Until this is fixed, let's not create new JIRAs for AcceptanceTest failures. -Dan On Tue, Aug 21,

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Jacob Barrett
Until docker on docker is supported for acceptance tests you can disable the parallelism on forks with -DdunitParallelForks=1 when running acceptanceTest. We can do the same in the CI for now too. :( The change for the CI can be found in ci/pipelines/shared/variablesomething.yml. -Jake On Tue,

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
I looked through the gradle files and couldn't figure out where to make the changes. If anyone else knows, please make the changes or let me know if you want to pair with me so I can learn. On Tue, Aug 21, 2018 at 11:32 AM, Anthony Baker wrote: > By design, acceptanceTest should be run with defa

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Anthony Baker
By design, acceptanceTest should be run with defaults (including ports). So if they aren’t run in containers they can’t be run in parallel. Anthony > On Aug 21, 2018, at 11:15 AM, Kirk Lund wrote: > > Oops... I mean "orphaned processes" > > On Tue, Aug 21, 2018 at 11:13 AM, Kirk Lund wrote

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
Oops... I mean "orphaned processes" On Tue, Aug 21, 2018 at 11:13 AM, Kirk Lund wrote: > So, we need to disable running them in parallel and improve the tearDown > to make sure these tests don't leave an orphaned tests behind. > > On Tue, Aug 21, 2018 at 11:11 AM, Sai Boorlagadda < > sai.boorlag

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
So, we need to disable running them in parallel and improve the tearDown to make sure these tests don't leave an orphaned tests behind. On Tue, Aug 21, 2018 at 11:11 AM, Sai Boorlagadda wrote: > jdbc-connector acceptance tests need docker-in-docker (also docker-compose) > to spin up mysql and po

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Sai Boorlagadda
jdbc-connector acceptance tests need docker-in-docker (also docker-compose) to spin up mysql and postgres. On Tue, Aug 21, 2018 at 11:04 AM Dan Smith wrote: > Actually, it looks like the problem is that we are *not* using docker > containers for the acceptance tests. Check this out, in > gradle/

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Jacob Barrett
The acceptance tests used to run fine in parallel in the same host. We need docker in docker working to containerize them. We would need to add docker to the docker image and some other fun stuff. Short term you can make the tests run serially. :( -Jake > On Aug 21, 2018, at 11:04 AM, Dan Smi

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
A good rule of thumb is to never test with default ports unless the test runs in a container. Otherwise, running tests in parallel will fail miserably. On Tue, Aug 21, 2018 at 10:44 AM, Kirk Lund wrote: > GEODE-5590 would seem to imply that GfshRule does not have an adequate > safe guard? If it

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Dan Smith
Actually, it looks like the problem is that we are *not* using docker containers for the acceptance tests. Check this out, in gradle/docker.gradle. Since acceptance tests use the default port, this means the test are guaranteed to be flaky, especially since we are running them in parallel: //A

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kenneth Howe
> On Aug 21, 2018, at 10:44 AM, Kirk Lund wrote: > > GEODE-5590 would seem to imply that GfshRule does not have an adequate safe > guard? If it spawns a server process which binds to the default server port > and that process persists after the test then we need better tearDown. > Yes, that d

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
GEODE-5590 would seem to imply that GfshRule does not have an adequate safe guard? If it spawns a server process which binds to the default server port and that process persists after the test then we need better tearDown. Actually I thought we were using Docker to run each AcceptanceTest in isola

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kenneth Howe
GEODE-5590 was opened recently for these tests that have started failing with BindExceptions on the default server port. > On Aug 21, 2018, at 10:27 AM, Kirk Lund wrote: > > Oh yeah, I see it now! I forgot how to read that page. Thanks! > > On Tue, Aug 21, 2018 at 10:18 AM, Sai Boorlagadda > w

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Jens Deppe
Recently all of the AcceptanceTest failures I've looked at have been around bind exceptions on startup which implies test pollution from prior tests. --Jens On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda wrote: > The metrics job themselves will be green (as they complete to success) but > you

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
Oh yeah, I see it now! I forgot how to read that page. Thanks! On Tue, Aug 21, 2018 at 10:18 AM, Sai Boorlagadda wrote: > The metrics job themselves will be green (as they complete to success) but > you can expand the get_metrics task output and see that build#20 started > reporting these failur

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Sai Boorlagadda
DeployWithLargeJarTest & PutCommandWithJsonTest are flaky on Develop. DeployWithLargeJarTest - https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/AcceptanceTest/builds/335 PutCommandWithJsonTest - https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/Acceptan

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Sai Boorlagadda
The metrics job themselves will be green (as they complete to success) but you can expand the get_metrics task output and see that build#20 started reporting these failures, so probably these are due to recent changes on develop. I believe these metrics are from develop CI test runs. On Tue, Aug 2

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Kirk Lund
Those metrics show AcceptanceTests consistently GREEN. Do these metrics include test failures from pull request precheckin runs like mine? Or does it just cover CI test runs? On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda wrote: > Metrics show these started failing recently. > > https://conco

Re: PutCommandWithJsonTest and DeployWithLargeJarTest

2018-08-21 Thread Sai Boorlagadda
Metrics show these started failing recently. https://concourse.apachegeode-ci.info/teams/main/pipelines/metrics/jobs/GeodeAcceptanceTestMetrics/builds/20 On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund wrote: > Are PutCommandWithJsonTest and DeployWithLargeJarTest known to be flaky? > > My latest pu