+1 to the new test organization. And I agree with Patrick's explanation of which is the more stable test category.
On Thu, Jun 28, 2018 at 9:47 AM, Patrick Rhomberg <prhomb...@pivotal.io> wrote: > @Dale > It seems to me that the Unit / Distributed / etc would be the more stable > purpose, and as importantly, each level is mutually-exclusive with any > other. > At least under our current categorization tags, the functional area > categories don't form a partition on our test set. For instance, the > GatewayLegacyAuthenticationRegressionTest is both a WanTest and a > SecurityTest. > > On Wed, Jun 27, 2018 at 10:19 AM, Kirk Lund <kl...@apache.org> wrote: > > > Don't forget to add "acceptanceTest/" > > > > On Tue, Jun 26, 2018 at 3:36 PM, Jacob Barrett <jbarr...@pivotal.io> > > wrote: > > > > > I'd like to suggest that we refactor our current test source set, which > > > contains both unit, integration and distributed tests, into distinct > > source > > > sets, test, integrationTest, distributedTest. These source sets would > > > replace the use of the primary categories UnitTest, IntegrationTest and > > > DistributedTest. > > > > > > The catalyst for this change is an issue that Gradle's test runner > > doesn't > > > pre-filter categories when launching tests, so if the tests are > launched > > in > > > separate JVMs or Docker containers, like :distributeTest task, the cost > > of > > > spinning up those resources is realized only to immediately exit > without > > > running any test for all test classes in the module. Switching to > > separate > > > source sets for each category would remove the need to filter on > category > > > and only tests in the corresponding source set would get executed in > > their > > > external JVM or Docker container. An example of this issue is > > > geode-junit:distributedTest task, which forks all test classes in > > separate > > > JVMs but never actually runs any tests since there are no > DistributedTest > > > tagged tests. > > > > > > The secondary effect is a way too isolate dependencies in each of the > > > source sets. Unit tests in the test set would not have dependencies > need > > > for integration tests or distributed test so that if you accidentally > > tried > > > to import classes from those frameworks you would get a compiler > failure. > > > Likewise, integration tests would not include distributed test > framework > > > dependencies. Any shared test classes like mock, dummies, fakes, etc. > > could > > > be shared in a commonTest source set, but would not contain any tests > > > itself. > > > > > > The proposed structure would look like this: > > > > > > test/ - only contains unit tests. > > > integrationTest/ - only contains integration style tests. > > > distributedTest/ - only includes DUnit based tests. > > > commonTest/ - includes commonly shared classes between each test > > category. > > > Does not contain any classes. > > > > > > Thoughts? > > > > > > -Jake > > > > > >
