Opened a ticket here https://issues.apache.org/jira/browse/GEODE-5363
On Thu, Jun 28, 2018 at 10:36 AM Galen O'Sullivan <gosulli...@apache.org> wrote: > +1 to the new test organization. > > And I agree with Patrick's explanation of which is the more stable test > category. > > On Thu, Jun 28, 2018 at 9:47 AM, Patrick Rhomberg <prhomb...@pivotal.io> > wrote: > > > @Dale > > It seems to me that the Unit / Distributed / etc would be the more > stable > > purpose, and as importantly, each level is mutually-exclusive with any > > other. > > At least under our current categorization tags, the functional area > > categories don't form a partition on our test set. For instance, the > > GatewayLegacyAuthenticationRegressionTest is both a WanTest and a > > SecurityTest. > > > > On Wed, Jun 27, 2018 at 10:19 AM, Kirk Lund <kl...@apache.org> wrote: > > > > > Don't forget to add "acceptanceTest/" > > > > > > On Tue, Jun 26, 2018 at 3:36 PM, Jacob Barrett <jbarr...@pivotal.io> > > > wrote: > > > > > > > I'd like to suggest that we refactor our current test source set, > which > > > > contains both unit, integration and distributed tests, into distinct > > > source > > > > sets, test, integrationTest, distributedTest. These source sets would > > > > replace the use of the primary categories UnitTest, IntegrationTest > and > > > > DistributedTest. > > > > > > > > The catalyst for this change is an issue that Gradle's test runner > > > doesn't > > > > pre-filter categories when launching tests, so if the tests are > > launched > > > in > > > > separate JVMs or Docker containers, like :distributeTest task, the > cost > > > of > > > > spinning up those resources is realized only to immediately exit > > without > > > > running any test for all test classes in the module. Switching to > > > separate > > > > source sets for each category would remove the need to filter on > > category > > > > and only tests in the corresponding source set would get executed in > > > their > > > > external JVM or Docker container. An example of this issue is > > > > geode-junit:distributedTest task, which forks all test classes in > > > separate > > > > JVMs but never actually runs any tests since there are no > > DistributedTest > > > > tagged tests. > > > > > > > > The secondary effect is a way too isolate dependencies in each of the > > > > source sets. Unit tests in the test set would not have dependencies > > need > > > > for integration tests or distributed test so that if you accidentally > > > tried > > > > to import classes from those frameworks you would get a compiler > > failure. > > > > Likewise, integration tests would not include distributed test > > framework > > > > dependencies. Any shared test classes like mock, dummies, fakes, etc. > > > could > > > > be shared in a commonTest source set, but would not contain any tests > > > > itself. > > > > > > > > The proposed structure would look like this: > > > > > > > > test/ - only contains unit tests. > > > > integrationTest/ - only contains integration style tests. > > > > distributedTest/ - only includes DUnit based tests. > > > > commonTest/ - includes commonly shared classes between each test > > > category. > > > > Does not contain any classes. > > > > > > > > Thoughts? > > > > > > > > -Jake > > > > > > > > > >
