Re: [PR] Make task executor non-final [lucene]

2025-05-01 Thread via GitHub
Shibi-bala commented on PR #14524: URL: https://github.com/apache/lucene/pull/14524#issuecomment-2845644300 @javanna I think what users need (or expect) with a TaskExecutor is to be able to have the created tasks run on the ExecutorService provided by the user. In my example, I'd like to be

Re: [PR] cache preset dict for LZ4WithPresetDictDecompressor [lucene]

2025-05-01 Thread via GitHub
github-actions[bot] commented on PR #14397: URL: https://github.com/apache/lucene/pull/14397#issuecomment-2846053145 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] [Bug] Fix for stored fields force merge regression [lucene]

2025-05-01 Thread via GitHub
github-actions[bot] commented on PR #14512: URL: https://github.com/apache/lucene/pull/14512#issuecomment-2846053069 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844540507 Hi, maybe we should improve the Java version check on Gradle startup a bit and disallow EA versions there, if the boot JDK major version is equal to minimum requirements. Some

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844545708 We could also disallow EA versions for Gradle in general. Of course you can still run the tests and compilation with EA versions using `RUNTIME_JAVA_HOME`. -- This is an automa

Re: [PR] Disallow EA versions to run Gradle [lucene]

2025-05-01 Thread via GitHub
uschindler merged PR #14601: URL: https://github.com/apache/lucene/pull/14601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

Re: [PR] Rewrite expressions module with Java 24 classfile API [lucene]

2025-05-01 Thread via GitHub
uschindler commented on PR #14597: URL: https://github.com/apache/lucene/pull/14597#issuecomment-2845352242 I made some followup changes here: #14602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Refactor the expressions compiler to use official ClassData BSM with indexed lookup [lucene]

2025-05-01 Thread via GitHub
uschindler merged PR #14602: URL: https://github.com/apache/lucene/pull/14602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

Re: [PR] Refactor the expressions compiler to use official ClassData BSM with indexed lookup [lucene]

2025-05-01 Thread via GitHub
uschindler commented on PR #14602: URL: https://github.com/apache/lucene/pull/14602#issuecomment-2845371799 @s1monw it should be same function, not similar! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844679676 Here is my idea: #1460#14595 I think this is simplest. It works fine, we can try it also on ASF Jenkins. -- This is an automated message from the Apache Git Service. To

Re: [PR] Add a Faiss codec for KNN searches [lucene]

2025-05-01 Thread via GitHub
mikemccand commented on PR #14178: URL: https://github.com/apache/lucene/pull/14178#issuecomment-2844751984 I was able to test this PR, yay! I first installed `faiss-cpu` via the Anaconda `pytorch` channel into my dev box, then applied the patch diff from this PR, then tweaked luceneu

[I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
dweiss opened a new issue, #14600: URL: https://github.com/apache/lucene/issues/14600 ### Description Specifically, Oracle OpenJDK 64-Bit Server VM 24-ea+2-133. This causes build failures on main because #14597 requires class builder API and it's declared as a feature in ea. #

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844243674 Oh thats horrible. I hope we can remove that workaround commit soon! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844247399 > Specifically, Oracle OpenJDK 64-Bit Server VM 24-ea+2-133. Thats the oldest one and is basically Java 23 Do you know why the JDK versions are so outdated espec

Re: [PR] Rewrite expressions module with Java 24 classfile API [lucene]

2025-05-01 Thread via GitHub
uschindler commented on PR #14597: URL: https://github.com/apache/lucene/pull/14597#issuecomment-2844245278 I made a small followup commit to make usage more like recommended by Classfile API docs: e2d06855cc09df4b1293817c99d767ef7d8adc6d -- This is an automated message from the Apache Gi

Re: [PR] Switch s2-geometry to google's 'official' version [lucene]

2025-05-01 Thread via GitHub
dweiss commented on PR #14599: URL: https://github.com/apache/lucene/pull/14599#issuecomment-2844257597 This requires pulling in guava, which in turn has a module-info in a multi-release jar location. ECJ fails on this due to this bug I filed 4 years ago: https://bugs.eclipse.org/bugs/sh

Re: [PR] Refactor the expressions compiler to use official ClassData BSM with indexed lookup [lucene]

2025-05-01 Thread via GitHub
uschindler commented on PR #14602: URL: https://github.com/apache/lucene/pull/14602#issuecomment-2845434333 The backport is the following commit: 6fb9dee0e2078a6044dd3a5c0020736f14fb2f5a -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Rewrite expressions module with Java 24 classfile API [lucene]

2025-05-01 Thread via GitHub
uschindler commented on PR #14597: URL: https://github.com/apache/lucene/pull/14597#issuecomment-2844596152 Finally here is an example Classfile output with this PR for this code: ```java public void testNamespacesWithoutDirectMH() throws Exception { Map functions =

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
dweiss commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844352546 It didn't work and I don't think it makes sense to invest any more time in this. I've reverted the hack and turned off jenkins builds until the infrastructure is up-to-date. -- Th

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844378552 I agree with this! From the mesage on the INFRA issue it seems that he did not yet update Jenkins JVMs, he just mentioned why lucene-solr-3 and -4 have no JDK 24 ("because

Re: [I] How to control the total number of merging threads when vector data merging easily leads to memory overflow and high CPU cost [lucene]

2025-05-01 Thread via GitHub
weizijun commented on issue #14554: URL: https://github.com/apache/lucene/issues/14554#issuecomment-2844451376 > We likely have the same problem in our env (Elasticsearch 8.18.0) This PR : https://github.com/apache/lucene/pull/14527 can significantly help reduce heap memory. -- Thi

Re: [PR] Support adaptive refresh in Searcher Managers. [lucene]

2025-05-01 Thread via GitHub
mikemccand commented on PR #14443: URL: https://github.com/apache/lucene/pull/14443#issuecomment-2844952688 > Sorry I'm still a bit confused: how is this approach better than just committing more frequently, replicating commits as soon as they are created, and refreshing searchers as soon a

Re: [PR] Support adaptive refresh in Searcher Managers. [lucene]

2025-05-01 Thread via GitHub
mikemccand commented on code in PR #14443: URL: https://github.com/apache/lucene/pull/14443#discussion_r2070345447 ## lucene/core/src/java/org/apache/lucene/search/SearcherManager.java: ## @@ -131,17 +133,32 @@ public SearcherManager(DirectoryReader reader, SearcherFactory sear

Re: [PR] Make task executor non-final [lucene]

2025-05-01 Thread via GitHub
javanna commented on PR #14524: URL: https://github.com/apache/lucene/pull/14524#issuecomment-2844725328 Having introduced TaskExecutor in the first place and worked quite a bit on inter-segment concurrency and other ways to parallelize execution, the general idea was to make concurrency as

Re: [PR] Add a Faiss codec for KNN searches [lucene]

2025-05-01 Thread via GitHub
kaivalnp commented on PR #14178: URL: https://github.com/apache/lucene/pull/14178#issuecomment-2844866562 Thank you for the benchmarks @mikemccand ! > slower the 11 segment case is (6.5 vs .9 msec) -- maybe the search is not concurrent across segments? Yes, it is sequential --

Re: [PR] Switch s2-geometry to google's 'official' version [lucene]

2025-05-01 Thread via GitHub
dweiss commented on PR #14599: URL: https://github.com/apache/lucene/pull/14599#issuecomment-2845276315 I've already excluded what was possible. Anything more would require copying the code over and manual removal of dependencies (which shouldn't be too difficult). I'm at crossroads here. T

Re: [PR] Add a Faiss codec for KNN searches [lucene]

2025-05-01 Thread via GitHub
mikemccand commented on PR #14178: URL: https://github.com/apache/lucene/pull/14178#issuecomment-2845297912 > > slower the 11 segment case is (6.5 vs .9 msec) -- maybe the search is not concurrent across segments? > > Yes, it is sequential -- we're not passing an executor [here](http

[PR] Refactors the Expressions compiler to use the official ClassData BSM for indexed lookup [lucene]

2025-05-01 Thread via GitHub
uschindler opened a new pull request, #14602: URL: https://github.com/apache/lucene/pull/14602 With more review of the code, I thought that it would be better to use one of the default Bootstrap methods available for ClassData. This allows to get rid of our custom BSM method and just use th

Re: [PR] Refactors the Expressions compiler to use the official ClassData BSM for indexed lookup [lucene]

2025-05-01 Thread via GitHub
uschindler commented on PR #14602: URL: https://github.com/apache/lucene/pull/14602#issuecomment-2845328503 I also added a new test to make sure that using the same function two times works correct. ```java public void testSameFunctionTwoTimes() throws Exception { Expressi

Re: [PR] Add a Faiss codec for KNN searches [lucene]

2025-05-01 Thread via GitHub
kaivalnp commented on PR #14178: URL: https://github.com/apache/lucene/pull/14178#issuecomment-2845342424 > but then multi-segment is only a bit slower (not proportionally so) I think this is because Lucene can take advantage of a shared queue of results, and stop searching in segment

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
dweiss commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844347781 I don't think the workaround will work at all. Let's see. I will revert the hack before closing this issue. -- This is an automated message from the Apache Git Service. To respond

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
dweiss commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844498810 I don't understand it either. Let's just wait. We have reasonable coverage with your jenkins and gh. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Switch s2-geometry to google's 'official' version [lucene]

2025-05-01 Thread via GitHub
dweiss commented on PR #14599: URL: https://github.com/apache/lucene/pull/14599#issuecomment-2844650917 I've made it run and compile. It is surprisingly hairy because of modular classpaths, errorprone interactions and extra dependencies brought in. There is a single version of this s2 geome

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
dweiss commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844619480 Another alternative is to embrace toolchains to download the base jdk for us, letting gradle run with whatever it is compatible with. Much like with the current hack for mrjars. --

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844551272 P.S.: Release candidate builds don't have `pre()` set, they behave like normal GA releases. In fact, they will be released later as GA. -- This is an automated message from the

Re: [PR] Fix tests: too many calls to IndexInput.clone during merging [lucene]

2025-05-01 Thread via GitHub
dweiss commented on code in PR #14595: URL: https://github.com/apache/lucene/pull/14595#discussion_r2070158622 ## lucene/core/src/test/org/apache/lucene/index/TestForTooMuchCloning.java: ## @@ -47,7 +50,13 @@ public void test() throws Exception { .setMaxBuffered

Re: [PR] Switch s2-geometry to google's 'official' version [lucene]

2025-05-01 Thread via GitHub
rmuir commented on PR #14599: URL: https://github.com/apache/lucene/pull/14599#issuecomment-2844658458 I like the idea of reducing the deps: not anything crazy, but just trying to keep the dep tree minimal. We frequently hear from users on decades-old versions on the ML, so it is less probl

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2844660289 Hi, yes and no. From the mrjars discussion I know that some people don't like it when Gradle downloads their JDK. For the regenerate task in the mrjar (or better said) vector cod

Re: [PR] Make TermStates#build concurrent [lucene]

2025-05-01 Thread via GitHub
navneet1v commented on code in PR #12183: URL: https://github.com/apache/lucene/pull/12183#discussion_r2070683991 ## lucene/core/src/java/org/apache/lucene/index/TermStates.java: ## @@ -86,19 +92,58 @@ public TermStates( * @param needsStats if {@code true} then all leaf cont

[PR] build(deps): bump ruff from 0.11.7 to 0.11.8 in /dev-tools/scripts [lucene]

2025-05-01 Thread via GitHub
dependabot[bot] opened a new pull request, #14603: URL: https://github.com/apache/lucene/pull/14603 Bumps [ruff](https://github.com/astral-sh/ruff) from 0.11.7 to 0.11.8. Release notes Sourced from https://github.com/astral-sh/ruff/releases";>ruff's releases. 0.11.8 Releas

Re: [I] Revert jenkins to use lucene label instead of sticky lucene-solr-1 [lucene]

2025-05-01 Thread via GitHub
dweiss commented on issue #14590: URL: https://github.com/apache/lucene/issues/14590#issuecomment-2846420381 I've enabled: * https://ci-builds.apache.org/job/Lucene/job/Lucene-Maven-Snapshots-main/ * https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-main to see if everything w

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
uschindler commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2845772648 Looks like lucene-solr-1 works correct. I triggered a job and it succeeded. Not sure about the other nodes. Uwe -- This is an automated message from the Apache Git Servi

Re: [I] Apache jenkins nodes use java 24-ea [lucene]

2025-05-01 Thread via GitHub
dweiss commented on issue #14600: URL: https://github.com/apache/lucene/issues/14600#issuecomment-2846399752 I see this in the output of [1]: ``` Eclipse Temurin OpenJDK Runtime Environment 24.0.1+9 ``` Looks ok to me. I'd wait until Gavin responds on the jira issue before en