[GitHub] [lucene] vsop-479 commented on pull request #11888: [Fix] Binary search the entries when all suffixes have the same length in a leaf block.
vsop-479 commented on PR #11888: URL: https://github.com/apache/lucene/pull/11888#issuecomment-1312467537 @jpountz I moved the test case for this bug to TestLucene90PostingsFormat. Please have a check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir opened a new issue, #11924: gradle build uses excessive resources on multi-core machine
rmuir opened a new issue, #11924: URL: https://github.com/apache/lucene/issues/11924 ### Description on e.g. an 8-core machine, gradle might spin up 4 jvms by default. but each jvm is ergonomically sized for an 8-core machine. so you get `8 * 4 = 32` "threads" or whatever it may be. solution, pass: `-XX:ActiveProcessorCount=1` to each jvm we fork, so that it sizes compiler, gc, threadpools appropriately. on my machine, i've been passing this via `org.gradle.jvmargs` and `tests.jvmargs` for over a year. but we need to default this on. it is still missing from some forked jvms (e.g. javadoc), and when these run, my computer becomes unresponsive because resources are allocated incorrectly. I opened this issue before, but I don't think I was able to communicate both the issue and how bad it is. Let's fix it this time. ### Version and environment details _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11924: gradle build uses excessive resources on multi-core machine
rmuir commented on issue #11924: URL: https://github.com/apache/lucene/issues/11924#issuecomment-1312478615 Basically this behavior grows `O(n)^2` where n is the number of cpus. The more you have, the more this is slowing you down. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11924: gradle build uses excessive resources on multi-core machine
rmuir commented on issue #11924: URL: https://github.com/apache/lucene/issues/11924#issuecomment-1312479438 one important thing when passing this flag, is to also configure the GC too, else you get serialgc on a 1-cpu jvm. But we should always be setting the JVM anyway (to ParallelGC) for builds. G1 is just throwing cpu cycles on the floor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir opened a new issue, #11925: error-prone's JVM arguments need help
rmuir opened a new issue, #11925: URL: https://github.com/apache/lucene/issues/11925 ### Description error-prone runs extremely slowly because on an N-core computer, it spins up N/2 jvms, each one ergonomically sized for N as far as compiler threads etc. And it uselessly enables C2 compiler and G1 garbage collector. when error-prone runs on my computer, the desktop gets unresponsive, can't navigate or click links or do anything else... it also has painfully slow execution due to these problems. we need to pass this thing the usual stuff at least: `-XX:+UseParallelGC -XX:TieredStopAtLevel=1 -XX:ActiveProcessorCount=1`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] madrob commented on issue #11924: gradle build uses excessive resources on multi-core machine
madrob commented on issue #11924: URL: https://github.com/apache/lucene/issues/11924#issuecomment-1312491533 That's a super interesting observation, do you think we should also report it to gradle? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] dweiss commented on issue #11924: gradle build uses excessive resources on multi-core machine
dweiss commented on issue #11924: URL: https://github.com/apache/lucene/issues/11924#issuecomment-1312502293 Depends what all those threads are doing... For I/O bound jobs they're idle most of the time. But I agree it's wasteful, it's fine to change the defaults. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11925: error-prone's JVM arguments need help
rmuir commented on issue #11925: URL: https://github.com/apache/lucene/issues/11925#issuecomment-1312638502 I stared at https://github.com/tbroyer/gradle-errorprone-plugin/blob/main/src/main/kotlin/net/ltgt/gradle/errorprone/ErrorPronePlugin.kt but I couldnt figure out yet how to tweak this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11924: gradle build uses excessive resources on multi-core machine
rmuir commented on issue #11924: URL: https://github.com/apache/lucene/issues/11924#issuecomment-1312640064 Yes, for background I think the flag was added with docker usages in mind: https://bugs.openjdk.org/browse/JDK-8189497 But the idea is the same here. Since we're making the build go parallel with multiple jvms (e.g. 4), let's prevent from overwhelming the machine. For this issue i would suggest adding `-XX:ActiveProcessorCount=1` to: * `gradle/hacks/turbocharge-jvm-opts.gradle` * `gradle/testing/defaults-tests.gradle` * `help/localSettings.txt` This way, the many forked jvms that we spin up only have smaller number of compiler/gc threads/default threadpool sizes each. The issue is especially exacerbated by #11925, because we're still enabling things like C2 compiler there, so the many forked error-prone-monkey-patched-javac tasks all waste a ton of cpu for no good reason. But currently I don't have enough gradle kung-fu to figure out how to pass jvm args to the error-prone :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] rmuir commented on pull request #11923: enable error-prone "narrow calculation" check
rmuir commented on PR #11923: URL: https://github.com/apache/lucene/pull/11923#issuecomment-1312641592 @uschindler @risdenk if you want to take another pass, it would be helpful. I don't want to introduce new risks here with this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
