gortiz commented on PR #11376: URL: https://github.com/apache/pinot/pull/11376#issuecomment-1683553753
The issue we have is that `report-aggregate`, by default, does only consider coverage by dependencies. That means that if project A depends on project B, lines covered by tests defined on project A do not count on `report-aggregate`. You can easily see that by reading `target/site/jacoco-aggregate/jacoco.csv` on any project after running `mvn test`. Instead, `report` takes into account the coverage of the tests on the project itself but not the tests on the dependencies. Why do we get smaller coverage with this? I'm not sure. But with this change we are actually having coverage in projects we didn't have before. For example, see the difference between `pinot-query-runtime` in [this commit](https://app.codecov.io/gh/apache/pinot/commit/85844dafb7e5e57f2fe71b7a7f019cdc6a9dc9eb/tree/pinot-query-runtime) vs what is in [master](https://app.codecov.io/gh/apache/pinot/tree/master/pinot-query-runtime). `report-aggregate` should be used in a single maven project that depends (directly or indirectly) on all other. But we want to split the test execution in two in order to have some parallelism (by doing so, we reduce the test time from 1h30 to 1h, aprox). This is why I also opened https://github.com/apache/pinot/pull/11375 in order to test what happens when we run all tests in the same aggregator (as seen there, the coverage increases but the time spent on tests increases as well). We can still have a single aggregation project that depends on different modules depending on the active profile, but we would need to list all projects instead of using negation expressions as we use in the current scripts. Ideally, what we should do is to run tests in parallel in the same reactor. There are two problems there: 1. It seems easier to do in junit than in testng. 2. Some tests may not be prepared to run concurrently (for example, integration tests that reserve ports). I think I'm going to create a GH issue to discuss about that in a broader way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
