Agreed on the branch merging. It’s been great to have it running the full set of tests!
> On Oct 5, 2023, at 10:58 PM, David Smiley <dsmi...@apache.org> wrote: > > I believe the Crave issues with branch merging seem to have been fixed. If > someone sees otherwise, please let me know. > > And boy Crave is fast! The whole GHA action takes 8m but Crave side is 6m of > which 4m of it is tests running. It's faster than "precommit" will is still > running in a standard GHA. Isn't that crazy! Yes, there's room for > improvement. > > There are opportunities for Crave to come up with a GHA self hosted runner to > substantially eat away at that 2m, like a needless checkout of all the code > on the GHA side that basically isn't used. > > There are opportunities for our project to try to optimize the Gradle build > so that it can start running tests (or whatever task) as soon as possible no > matter where it runs. There's a whole section to the Gradle docs on build > optimization. Maybe someone would like to explore that, like trying the > "configuration cache" > https://docs.gradle.org/current/userguide/configuration_cache.html > > I have access to build analytics in Crave that give some insights: The first > 48 seconds is not very concurrent and not downloading anything. The next 36 > seconds it downloads 100MB of something (don't know what). Then CPUs go full > tilt with tests. It's very apparent that Gradle testing has no "work > stealing" algorithm amongst the runners. > > > > I'm a bit perplexed at the downloading of 100MB because the image for the > build machine has commands I added to pre-download stuff. That looks like > the following: > > # Pre-download what we can through Gradle > ./gradlew --write-verification-metadata sha256 --dry-run > rm gradle/verification-metadata.dryrun.xml > ./gradlew -p solr/solr-ref-guide downloadAntora > ./gradlew -p solr/packaging downloadBats > # May need more memory > sed -i 's/-Xmx1g/-Xmx2g/g' gradle.properties > # Use lots of CPUs > sed -i 's/org.gradle.workers.max=.*/org.gradle.workers.max=96/' > gradle.properties > sed -i 's/tests.jvms=.*/tests.jvms=96/' gradle.properties > > ./gradlew assemble || true > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
