Usually you will endup with one engine and one or more test APIs. The documentation above solves all combinations with junit4, junit5 and testng. If somebody uses one @Test annotation and wants to say that f/w1 should run test class 1 but f/w2 should run test class 2, then the plugin should be configured with two executions within one phase default-test. I don't think we should rewrap entire plugin config params per provider since it can be solved with Maven's <executions/> in POM. T
On Wed, Nov 3, 2021 at 6:22 PM Slawomir Jaranowski <[email protected]> wrote: > Hi, > > I understand the difference between framework and providers. > I also know that we can add specific providers as dependency to the plugin > and in the result we have manually selected providers. > > When we don't configure any dependency the provider is chosen automatically > depending on what we have on classpath, what framework. > > eg. When we have on classpath junit4 and junit5, but without junit5 vintage > - junit4 tests are silently skipped. > > In most cases in the project we use one testing framework so > automatically choosing one provider is ok. > > When we use more that one framework it is probably during migration, so we > should define proper dependency. > It is ok when we have control on what we do in a project, but there are > some situations when we can make mistakes, like > - user add second testing framework and forgot about configuration > - we change project dependency and in transitive dependency we got next > testing framework > > So I want to mitigate the risk of skipped tests by warning / failing that > we have many frameworks and automatically provider selection takes place. > > We can also try to choose many providers (not one) automatically, but I > think it will be more complicated and in standard usage is not needed. > For automatic detection we have many corner cases, like testng + junit4 - > testng by default runs junit4 tests, so without special configuration for > testng we run some of the tests twice. > > > śr., 3 lis 2021 o 15:09 Tibor Digana <[email protected]> napisał(a): > > > Hi Slawomir, > > > > The reality is different. > > It's not the frameworks but providers. > > So the plugin iterates over providers. > > You may choose whether you will define the provider artifacts in plugin > > deps or you let the plugin to select the providers upon dependencies > which > > means that there is some logic and JUnit5 deps may activate one provider > > for both the junit4 and 5. > > > > T > > > > On Tue, Nov 2, 2021 at 7:46 PM Slawomir Jaranowski < > [email protected] > > > > > wrote: > > > > > Hi, > > > > > > When multiple test frameworks are present on project classpath surefire > > > chooses one framework for running the tests. > > > > > > In such situations, some tests are silently skipped. > > > > > > > > > I've created issue [1] for it with a simple proposition. > > > > > > After discussion on slack another proposition was in place to detect > the > > > test framework with other dependencies and choose a proper provider. > > > > > > Or use multiple providers instead of only one. > > > > > > > > > It will be good to choose how to solve this problem. > > > > > > > > > [1] https://issues.apache.org/jira/browse/SUREFIRE-1954 > > > > > > -- > > > Sławomir Jaranowski > > > > > > https://twitter.com/SlawekJaran > > > https://github.com/slawekjaranowski > > > https://linkedin.com/in/slawomirjaranowski > > > > > > > > -- > Sławomir Jaranowski >
