[ 
https://issues.apache.org/jira/browse/LUCENE-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306526#comment-17306526
 ] 

Dawid Weiss commented on LUCENE-9861:
-------------------------------------

Whaaat?! :) I'll take a look tomorrow, Robert but it looks terrific. We can 
definitely tune JVM defaults for these processes; I didn't realize the defaults 
are soo poor here!

You should really be looking at all dependencies of the "check -x test" task 
(checks without tests). Precommit is just a subset of those checks I typically 
run.

As for javadoc - I had this crazy idea that it'd be great to combine multiple 
doclets into a single pass over the sources. Javadoc is repeating the job of 
javac (and multiple javadoc passes are doing it multiple time). It'd be great 
to reuse the compile tree somehow. Don't know how to do it but it sounds super 
hacky... I mean, exciting.

> speed up precommit
> ------------------
>
>                 Key: LUCENE-9861
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9861
>             Project: Lucene - Core
>          Issue Type: Task
>            Reporter: Robert Muir
>            Priority: Major
>         Attachments: LUCENE-9861_hack.patch
>
>
> A lot of the java tools for precommit aren't being called in efficient ways 
> (compilation, linting, etc).
> For example ecjlint, it runs very slow:
> {noformat}
> Aggregate task times (possibly running in parallel!):
>  271.73 sec.  ecjLintMain
>  270.18 sec.  ecjLintTest
>  227.17 sec.  compileJava
>   12.07 sec.  compileTestJava
>    1.21 sec.  processResources
>    0.18 sec.  clean
> {noformat}
> Simplying adding a couple reasonable jvm arguments to the ecj linter 
> ({{jvmArgs = [ '-XX:+UseParallelGC', '-XX:TieredStopAtLevel=1' ]}}) speeds it 
> up significantly.
> Speedup for ecjLint is 3x for me:
> {noformat}
> Aggregate task times (possibly running in parallel!):
>  163.38 sec.  compileJava
>   84.57 sec.  ecjLintMain
>   83.12 sec.  ecjLintTest
>    6.11 sec.  compileTestJava
>    0.95 sec.  processResources
>    0.15 sec.  clean
> {noformat}
> I imagine same may be true for a lot of these tasks. We're currently tossing 
> default jvm args at these short-lived subprocesses, which is very suboptimal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to