[GitHub] [lucene] LuXugang merged pull request #12153: Unrelated code in TestIndexSortSortedNumericDocValuesRangeQuery

2023-03-15 Thread via GitHub
LuXugang merged PR #12153: URL: https://github.com/apache/lucene/pull/12153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.ap

[GitHub] [lucene] s1monw commented on a diff in pull request #12198: Reduce contention in DocumentsWriterFlushControl.

2023-03-15 Thread via GitHub
s1monw commented on code in PR #12198: URL: https://github.com/apache/lucene/pull/12198#discussion_r1136693360 ## lucene/core/src/java/org/apache/lucene/index/DocumentsWriterFlushControl.java: ## @@ -185,8 +185,12 @@ private boolean updatePeaks(long delta) { return true;

[GitHub] [lucene] s1monw commented on a diff in pull request #12199: Reduce contention in DocumentsWriterPerThreadPool.

2023-03-15 Thread via GitHub
s1monw commented on code in PR #12199: URL: https://github.com/apache/lucene/pull/12199#discussion_r1136706886 ## lucene/core/src/java/org/apache/lucene/index/ConcurrentApproximatePriorityQueue.java: ## @@ -0,0 +1,140 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [lucene] jpountz commented on pull request #12114: Use radix sort to sort postings when index sorting is enabled.

2023-03-15 Thread via GitHub
jpountz commented on PR #12114: URL: https://github.com/apache/lucene/pull/12114#issuecomment-1469650532 I purposedly introduced a bug to see what would fail, and only high-level tests that check early query termination or dynamic pruning failed, so I introduced lower-level tests that make

[GitHub] [lucene] jpountz merged pull request #12198: Reduce contention in DocumentsWriterFlushControl.

2023-03-15 Thread via GitHub
jpountz merged PR #12198: URL: https://github.com/apache/lucene/pull/12198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

[GitHub] [lucene] jpountz merged pull request #12114: Use radix sort to sort postings when index sorting is enabled.

2023-03-15 Thread via GitHub
jpountz merged PR #12114: URL: https://github.com/apache/lucene/pull/12114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

[GitHub] [lucene] mikemccand commented on issue #12185: Using DirectIODirectory results in BufferOverflowException

2023-03-15 Thread via GitHub
mikemccand commented on issue #12185: URL: https://github.com/apache/lucene/issues/12185#issuecomment-1469863743 Hmm, I think the root cause exception is essentially [thrown from here](https://github.com/openjdk/jdk17/blob/master/src/java.base/share/classes/java/nio/Buffer.java#L722), which

[GitHub] [lucene] jpountz merged pull request #12199: Reduce contention in DocumentsWriterPerThreadPool.

2023-03-15 Thread via GitHub
jpountz merged PR #12199: URL: https://github.com/apache/lucene/pull/12199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

[GitHub] [lucene] jpountz opened a new pull request, #12205: Remove remaining sources of contention on indexing.

2023-03-15 Thread via GitHub
jpountz opened a new pull request, #12205: URL: https://github.com/apache/lucene/pull/12205 With this change, running `IndexGeoNames` with 20 threads goes from 16-17 seconds to 15-16 seconds. If I disable the 3 text fields - which are the main bottleneck for indexing - out of 19 fields, the

[GitHub] [lucene] jpountz opened a new pull request, #12206: Fully reuse postings enums when flushing sorted indexes.

2023-03-15 Thread via GitHub
jpountz opened a new pull request, #12206: URL: https://github.com/apache/lucene/pull/12206 Currently we're only half reusing postings enums when flushing sorted indexes as we still create new wrapper instances every time, which can be costly with fields that have many terms. -- This is

[GitHub] [lucene] jpountz commented on a diff in pull request #12206: Fully reuse postings enums when flushing sorted indexes.

2023-03-15 Thread via GitHub
jpountz commented on code in PR #12206: URL: https://github.com/apache/lucene/pull/12206#discussion_r1137412879 ## lucene/core/src/java/org/apache/lucene/store/ByteBuffersDataInput.java: ## @@ -71,7 +71,11 @@ public ByteBuffersDataInput(List buffers) { this.blockMask = (1

[GitHub] [lucene] mdmarshmallow commented on pull request #12194: [GITHUB-11915] [Discussion Only] Make Lucene smarter about long runs of matches via new API on DISI

2023-03-15 Thread via GitHub
mdmarshmallow commented on PR #12194: URL: https://github.com/apache/lucene/pull/12194#issuecomment-1470400045 I tested with wikimedium10m. Looks like my change caused the `Prefix3` test to slow down.. not sure why. ``` TaskQPS baseline StdDevQPS my_mod

[GitHub] [lucene] jpountz commented on a diff in pull request #12194: [GITHUB-11915] [Discussion Only] Make Lucene smarter about long runs of matches via new API on DISI

2023-03-15 Thread via GitHub
jpountz commented on code in PR #12194: URL: https://github.com/apache/lucene/pull/12194#discussion_r1137546969 ## lucene/core/src/java/org/apache/lucene/search/DocIdSetIterator.java: ## @@ -211,4 +216,22 @@ protected final int slowAdvance(int target) throws IOException { *

[GitHub] [lucene] jpountz commented on a diff in pull request #12194: [GITHUB-11915] [Discussion Only] Make Lucene smarter about long runs of matches via new API on DISI

2023-03-15 Thread via GitHub
jpountz commented on code in PR #12194: URL: https://github.com/apache/lucene/pull/12194#discussion_r1137550527 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90PostingsReader.java: ## @@ -479,6 +481,31 @@ private void refillDocs() throws IOException { as

[GitHub] [lucene] jpountz commented on pull request #12194: [GITHUB-11915] [Discussion Only] Make Lucene smarter about long runs of matches via new API on DISI

2023-03-15 Thread via GitHub
jpountz commented on PR #12194: URL: https://github.com/apache/lucene/pull/12194#issuecomment-1470514750 > Do you have any pointer which benchmark task I could potentially use? If there isn't one available, I could try to add some next. Maybe we could try to leverage the geonames data

[GitHub] [lucene] zhaih merged pull request #12126: Refactor part of IndexFileDeleter and ReplicaFileDeleter into a common utility class

2023-03-15 Thread via GitHub
zhaih merged PR #12126: URL: https://github.com/apache/lucene/pull/12126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apach

[GitHub] [lucene] zhaih closed issue #11885: Refactor and generalize file deleter

2023-03-15 Thread via GitHub
zhaih closed issue #11885: Refactor and generalize file deleter URL: https://github.com/apache/lucene/issues/11885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib