gortiz commented on issue #11382: URL: https://github.com/apache/pinot/issues/11382#issuecomment-1683970732
> Side question, it looks like parallelizing the tests gave us a 30% improvement in time to run tests. Do you think we could split tests into 4 sets and get further improvement on test time? I don't like the idea of splitting test even more for two reasons: 1. It is a bit expensive (in terms of developer time) to balance how to split the tests between executors 2. Each executor consumes resources on GH. I don't now who pay for the GH executors or how many we have. But I'm assuming there are a few executors available. Consuming more executors per PR (theoretically) implies to have a bottleneck between PRs. I also don't know how large are the GH executors we are using, but given that we run maven with a parallelism of 16 I would guess they are quite large. I think it would be better to invest that developer time into the last proposed solution: to run test concurrently in a single maven reactor (and therefore in a single GH executor). @xiangfu0 is the one that has been working the most in balancing the test sets and I'm sure he knows more about the infrastructure we use. With that context, he may be more able to answer your question. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
