[
https://issues.apache.org/jira/browse/LUCENE-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306462#comment-17306462
]
Robert Muir commented on LUCENE-9861:
-------------------------------------
simple bench of current hack patch
before:
{noformat}
$ ./gradlew clean precommit
BUILD SUCCESSFUL in 6m 20s
581 actionable tasks: 514 executed, 67 up-to-date
{noformat}
after:
{noformat}
$ ./gradlew clean precommit
BUILD SUCCESSFUL in 3m 55s
581 actionable tasks: 514 executed, 67 up-to-date
{noformat}
For reference, here are the task times from the output:
{noformat}
Aggregate task times (possibly running in parallel!):
194.10 sec. compileTestJava
185.43 sec. renderJavadoc
184.49 sec. compileJava
85.09 sec. ecjLintMain
81.87 sec. spotlessJava
78.61 sec. ecjLintTest
29.43 sec. validateSourcePatterns
7.77 sec. rat
7.41 sec. jar
3.07 sec. compileToolsJava
2.66 sec. ecjLintTools
1.52 sec. verifyLocks
1.28 sec. processResources
0.86 sec. copyTestResources
0.75 sec. gitStatus
0.75 sec. validateJarChecksums
0.44 sec. collectJarInfos
0.15 sec. spotlessInternalRegisterDependencies
0.12 sec. clean
0.09 sec. validateJarLicenses
0.07 sec. processTestResources
0.05 sec. syncConf
0.04 sec. spotlessJavaCheck
0.02 sec. checkDanglingLicenseFiles
0.01 sec. checkWorkingCopyClean
0.01 sec. versionsPropsAreSorted
{noformat}
So I still need to investigate {{javac}}, {{spotlessJava}}, and
{{validateSourcePatterns}}
> speed up precommit
> ------------------
>
> Key: LUCENE-9861
> URL: https://issues.apache.org/jira/browse/LUCENE-9861
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Robert Muir
> Priority: Major
> Attachments: LUCENE-9861_hack.patch
>
>
> A lot of the java tools for precommit aren't being called in efficient ways
> (compilation, linting, etc).
> For example ecjlint, it runs very slow:
> {noformat}
> Aggregate task times (possibly running in parallel!):
> 271.73 sec. ecjLintMain
> 270.18 sec. ecjLintTest
> 227.17 sec. compileJava
> 12.07 sec. compileTestJava
> 1.21 sec. processResources
> 0.18 sec. clean
> {noformat}
> Simplying adding a couple reasonable jvm arguments to the ecj linter
> ({{jvmArgs = [ '-XX:+UseParallelGC', '-XX:TieredStopAtLevel=1' ]}}) speeds it
> up significantly.
> Speedup for ecjLint is 3x for me:
> {noformat}
> Aggregate task times (possibly running in parallel!):
> 163.38 sec. compileJava
> 84.57 sec. ecjLintMain
> 83.12 sec. ecjLintTest
> 6.11 sec. compileTestJava
> 0.95 sec. processResources
> 0.15 sec. clean
> {noformat}
> I imagine same may be true for a lot of these tasks. We're currently tossing
> default jvm args at these short-lived subprocesses, which is very suboptimal.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]