Re: [DISCUSS] When is a test not flaky anymore?

2018-07-10 Thread Patrick Rhomberg
Belatedly... During our investigation into the application of the FlakyTest category, we ran the FlakyTest precheckin target approximately 1,500 times over this past weekend. An abundance of these tests passed without failure. In my opinion, this provides the base confidence in these tests, an

Re: [DISCUSS] When is a test not flaky anymore?

2018-07-10 Thread Kirk Lund
FYI, most of the intermittent failures I've reviewed and fixed were caused by a Java GC during a small window in which the test assumes something asynchronous will complete without using Awaitility. So these tests are still inherently flaky if you don't change them. Unless you can find the async ra

Article and paper about Flaky tests

2018-07-10 Thread Kirk Lund
Here are two good resources about Flaky tests: How to Deal With and Eliminate Flaky Tests [1] An Empirical Analysis of Flaky Tests [2] [1] https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests [2] http://mir.cs.illinois.edu/~qluo2/fse14LuoHEM.pdf

Geode unit tests completed in 'develop/UITests' with non-zero exit code

2018-07-10 Thread apachegeodeci
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/UITests/builds/144

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #974 has FAILED

2018-07-10 Thread Spring CI
--- Spring Data GemFire > Nightly-ApacheGeode > #974 failed. --- Scheduled No failed tests found, a possible compilation error. https://build.spring.io/browse/SG

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #974 has FAILED

2018-07-10 Thread Spring CI
--- Spring Data GemFire > Nightly-ApacheGeode > #974 failed (rerun once). --- This build was rerun by John Blum. No failed tests found, a possible compilation err

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #974 was SUCCESSFUL (with 2423 tests)

2018-07-10 Thread Spring CI
--- Spring Data GemFire > Nightly-ApacheGeode > #974 was successful (rerun 2 times). --- This build was rerun by John Blum. 2425 tests in total. https://build.sp