@Dale It seems to me that the Unit / Distributed / etc would be the more stable purpose, and as importantly, each level is mutually-exclusive with any other. At least under our current categorization tags, the functional area categories don't form a partition on our test set. For instance, the GatewayLegacyAuthenticationRegressionTest is both a WanTest and a SecurityTest.
On Wed, Jun 27, 2018 at 10:19 AM, Kirk Lund <kl...@apache.org> wrote: > Don't forget to add "acceptanceTest/" > > On Tue, Jun 26, 2018 at 3:36 PM, Jacob Barrett <jbarr...@pivotal.io> > wrote: > > > I'd like to suggest that we refactor our current test source set, which > > contains both unit, integration and distributed tests, into distinct > source > > sets, test, integrationTest, distributedTest. These source sets would > > replace the use of the primary categories UnitTest, IntegrationTest and > > DistributedTest. > > > > The catalyst for this change is an issue that Gradle's test runner > doesn't > > pre-filter categories when launching tests, so if the tests are launched > in > > separate JVMs or Docker containers, like :distributeTest task, the cost > of > > spinning up those resources is realized only to immediately exit without > > running any test for all test classes in the module. Switching to > separate > > source sets for each category would remove the need to filter on category > > and only tests in the corresponding source set would get executed in > their > > external JVM or Docker container. An example of this issue is > > geode-junit:distributedTest task, which forks all test classes in > separate > > JVMs but never actually runs any tests since there are no DistributedTest > > tagged tests. > > > > The secondary effect is a way too isolate dependencies in each of the > > source sets. Unit tests in the test set would not have dependencies need > > for integration tests or distributed test so that if you accidentally > tried > > to import classes from those frameworks you would get a compiler failure. > > Likewise, integration tests would not include distributed test framework > > dependencies. Any shared test classes like mock, dummies, fakes, etc. > could > > be shared in a commonTest source set, but would not contain any tests > > itself. > > > > The proposed structure would look like this: > > > > test/ - only contains unit tests. > > integrationTest/ - only contains integration style tests. > > distributedTest/ - only includes DUnit based tests. > > commonTest/ - includes commonly shared classes between each test > category. > > Does not contain any classes. > > > > Thoughts? > > > > -Jake > > >
