[I] Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler [lucene]

2024-07-07 Thread via GitHub
aoli-al opened a new issue, #13547: URL: https://github.com/apache/lucene/issues/13547 ### Description The test `testSubclassConcurrentMergeScheduler` is flaky due to concurrency issues and produces following assertion failure: ``` java.lang.AssertionError at __ran

Re: [PR] Optimize MaxScoreBulkScorer [lucene]

2024-07-07 Thread via GitHub
jpountz commented on code in PR #13544: URL: https://github.com/apache/lucene/pull/13544#discussion_r1667761515 ## lucene/core/src/java/org/apache/lucene/search/MaxScoreBulkScorer.java: ## @@ -335,8 +334,11 @@ boolean partitionScorers() { System.arraycopy(allScorers, 0, scr

Re: [PR] Override single byte writes to OutputStreamIndexOutput to remove locking [lucene]

2024-07-07 Thread via GitHub
jpountz commented on code in PR #13543: URL: https://github.com/apache/lucene/pull/13543#discussion_r1667763097 ## lucene/core/src/java/org/apache/lucene/store/OutputStreamIndexOutput.java: ## @@ -135,5 +135,19 @@ void writeLong(long i) throws IOException { BitUtil.VH_LE_

Re: [PR] Use a confined Arena for IOContext.READONCE [lucene]

2024-07-07 Thread via GitHub
jpountz commented on code in PR #13535: URL: https://github.com/apache/lucene/pull/13535#discussion_r1667767007 ## lucene/core/src/java/org/apache/lucene/index/ReadersAndUpdates.java: ## @@ -578,7 +578,8 @@ public synchronized boolean writeFieldUpdates( // IndexWriter.com

Re: [PR] Use a confined Arena for IOContext.READONCE [lucene]

2024-07-07 Thread via GitHub
jpountz commented on PR #13535: URL: https://github.com/apache/lucene/pull/13535#issuecomment-2212581169 >> I'm contemplating making MockDirectoryWrapper also fail if IndexInputs get used in a different thread from the one where they were created. > > You mean this for READONCE ones, c

Re: [PR] WIP: draft of intra segment concurrency [lucene]

2024-07-07 Thread via GitHub
jpountz commented on PR #13542: URL: https://github.com/apache/lucene/pull/13542#issuecomment-2212586823 Thanks for looking into this! > IndexSearcher#count / TotalHitCountCollector rely on Weight#count(LeafReaderContext), which now gets called multiple times against the same leaf an

Re: [PR] TaskExecutor should not fork unnecessarily [lucene]

2024-07-07 Thread via GitHub
jpountz commented on PR #13472: URL: https://github.com/apache/lucene/pull/13472#issuecomment-2212589053 Should we now update `luceneutil`'s `TaskThreads` to fork queries into the `IndexSearcher`'s executor (when present)? -- This is an automated message from the Apache Git Service. To re

[PR] Refactor and javadoc update for KNN vector writer classes [lucene]

2024-07-07 Thread via GitHub
zhaih opened a new pull request, #13548: URL: https://github.com/apache/lucene/pull/13548 ### Description * Refactor logic of translating old ord to new ord * Javadoc for Lucene99HnswVectorsFormat is a bit outdated after we separate FlatVectorsFormat out, updated -- This

Re: [PR] TaskExecutor should not fork unnecessarily [lucene]

2024-07-07 Thread via GitHub
original-brownbear commented on PR #13472: URL: https://github.com/apache/lucene/pull/13472#issuecomment-2212623631 @jpountz yea makes sense to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Override single byte writes to OutputStreamIndexOutput to remove locking [lucene]

2024-07-07 Thread via GitHub
original-brownbear commented on code in PR #13543: URL: https://github.com/apache/lucene/pull/13543#discussion_r1667797299 ## lucene/core/src/java/org/apache/lucene/store/OutputStreamIndexOutput.java: ## @@ -135,5 +135,19 @@ void writeLong(long i) throws IOException { Bit

Re: [PR] Optimize MaxScoreBulkScorer [lucene]

2024-07-07 Thread via GitHub
original-brownbear commented on code in PR #13544: URL: https://github.com/apache/lucene/pull/13544#discussion_r1667800825 ## lucene/core/src/java/org/apache/lucene/search/MaxScoreBulkScorer.java: ## @@ -335,8 +334,11 @@ boolean partitionScorers() { System.arraycopy(allScor

Re: [PR] Optimize MaxScoreBulkScorer [lucene]

2024-07-07 Thread via GitHub
original-brownbear commented on PR #13544: URL: https://github.com/apache/lucene/pull/13544#issuecomment-2212636552 Thanks Adrien! > do you have a benchmark / profiler output that shows the improvement? The nightlies show this: ``` 0.49% 4090M java.lan

Re: [PR] Move synonym map off-heap for SynonymGraphFilter [lucene]

2024-07-07 Thread via GitHub
dungba88 commented on PR #13054: URL: https://github.com/apache/lucene/pull/13054#issuecomment-2212975749 @msfroh I also forgot about this. Let me create a PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Move synonym map off-heap for SynonymGraphFilter [lucene]

2024-07-07 Thread via GitHub
dungba88 commented on PR #13054: URL: https://github.com/apache/lucene/pull/13054#issuecomment-2213024535 I published a PR here: https://github.com/apache/lucene/pull/13549. Please take a look when you have time! -- This is an automated message from the Apache Git Service. To respond to t