[GitHub] [lucene] javanna commented on a diff in pull request #12523: TaskExecutor waits for all tasks to complete before returning

2023-09-05 Thread via GitHub
javanna commented on code in PR #12523: URL: https://github.com/apache/lucene/pull/12523#discussion_r1315463865 ## lucene/core/src/java/org/apache/lucene/search/TaskExecutor.java: ## @@ -50,16 +51,21 @@ final List invokeAll(Collection> tasks) { for (Runnable task : tasks)

[GitHub] [lucene] javanna commented on a diff in pull request #12516: Unwrap execution exceptions cause and rethrow as is when possible

2023-09-05 Thread via GitHub
javanna commented on code in PR #12516: URL: https://github.com/apache/lucene/pull/12516#discussion_r1315559947 ## lucene/core/src/java/org/apache/lucene/search/TaskExecutor.java: ## @@ -57,6 +58,12 @@ final List invokeAll(Collection> tasks) { } catch (InterruptedExcept

[GitHub] [lucene] javanna commented on pull request #12515: Offload single slice to executor

2023-09-05 Thread via GitHub
javanna commented on PR #12515: URL: https://github.com/apache/lucene/pull/12515#issuecomment-1706176412 I added the entry to the changelog, I now have a test failure around exception handling which is going to be addressed by #12516 , so I'd merge that one first and only then this PR. --

[GitHub] [lucene] javanna commented on a diff in pull request #12516: Unwrap execution exceptions cause and rethrow as is when possible

2023-09-05 Thread via GitHub
javanna commented on code in PR #12516: URL: https://github.com/apache/lucene/pull/12516#discussion_r1315608401 ## lucene/core/src/java/org/apache/lucene/search/AbstractKnnVectorQuery.java: ## @@ -95,20 +95,17 @@ public Query rewrite(IndexSearcher indexSearcher) throws IOExcept

[GitHub] [lucene] mikemccand commented on issue #12536: Remove `lastPosBlockOffset` from term metadata for Lucene90PostingsFormat

2023-09-05 Thread via GitHub
mikemccand commented on issue #12536: URL: https://github.com/apache/lucene/issues/12536#issuecomment-1706251517 Sorry, what went wrong when you tried to remove `lastPosBlockOffset`? Skipping is (currently) only at every 8 blocks boundaries right? So doesn't the skip data already know how

[GitHub] [lucene] jpountz commented on a diff in pull request #12516: Unwrap execution exceptions cause and rethrow as is when possible

2023-09-05 Thread via GitHub
jpountz commented on code in PR #12516: URL: https://github.com/apache/lucene/pull/12516#discussion_r1315641242 ## lucene/core/src/java/org/apache/lucene/search/TaskExecutor.java: ## @@ -57,6 +58,12 @@ final List invokeAll(Collection> tasks) { } catch (InterruptedExcept

[GitHub] [lucene] mikemccand commented on issue #12513: Try out a tantivy's term dictionary format

2023-09-05 Thread via GitHub
mikemccand commented on issue #12513: URL: https://github.com/apache/lucene/issues/12513#issuecomment-1706283574 > Yes, I actually tried to use FSTPostingsFormat in the benchmarks game and I had to increase the heap size from 4g to 32g to workaround the in-heap memory demand. Do you

[GitHub] [lucene] mikemccand commented on issue #12513: Try out a tantivy's term dictionary format

2023-09-05 Thread via GitHub
mikemccand commented on issue #12513: URL: https://github.com/apache/lucene/issues/12513#issuecomment-1706307210 > 2. random-addressing term information given an ordinal. again no additional scan; Hmm indeed this would require a fixed block size for every term's metadata. Does

[GitHub] [lucene] SevenCss commented on issue #7820: CheckIndex cannot "fix" indexes that have individual segments with missing or corrupt .si files because sanity checks will fail trying to read the

2023-09-05 Thread via GitHub
SevenCss commented on issue #7820: URL: https://github.com/apache/lucene/issues/7820#issuecomment-1706310673 @mikemccand Appreciated for your response. Glad to know that. I will have a try. :) -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [lucene] fulmicoton commented on issue #12513: Try out a tantivy's term dictionary format

2023-09-05 Thread via GitHub
fulmicoton commented on issue #12513: URL: https://github.com/apache/lucene/issues/12513#issuecomment-1706314422 > Does Tantivy do pulsing (inlining postings for a singleton terms into the terms dictionary)? No but we should. It has been on my task list for a long time. -- This

[GitHub] [lucene] SevenCss commented on issue #7820: CheckIndex cannot "fix" indexes that have individual segments with missing or corrupt .si files because sanity checks will fail trying to read the

2023-09-05 Thread via GitHub
SevenCss commented on issue #7820: URL: https://github.com/apache/lucene/issues/7820#issuecomment-1706323759 > > Third off, there is possibly a separate improvement we could make to IndexWriter, to remove segments_N files before removing all other files when a commit point is deleted, to tr

[GitHub] [lucene] javanna commented on a diff in pull request #12516: Unwrap execution exceptions cause and rethrow as is when possible

2023-09-05 Thread via GitHub
javanna commented on code in PR #12516: URL: https://github.com/apache/lucene/pull/12516#discussion_r1315852459 ## lucene/core/src/java/org/apache/lucene/search/TaskExecutor.java: ## @@ -57,6 +58,12 @@ final List invokeAll(Collection> tasks) { } catch (InterruptedExcept

[GitHub] [lucene] javanna merged pull request #12516: Unwrap execution exceptions cause and rethrow as is when possible

2023-09-05 Thread via GitHub
javanna merged PR #12516: URL: https://github.com/apache/lucene/pull/12516 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

[GitHub] [lucene] javanna merged pull request #12515: Offload single slice to executor

2023-09-05 Thread via GitHub
javanna merged PR #12515: URL: https://github.com/apache/lucene/pull/12515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

[GitHub] [lucene] javanna closed issue #12498: Simplify task executor for concurrent operations

2023-09-05 Thread via GitHub
javanna closed issue #12498: Simplify task executor for concurrent operations URL: https://github.com/apache/lucene/issues/12498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [lucene] romseygeek commented on issue #12538: clarify QueryVisitor.visitLeaf interaction with QueryVisitor.acceptField

2023-09-05 Thread via GitHub
romseygeek commented on issue #12538: URL: https://github.com/apache/lucene/issues/12538#issuecomment-1706747711 I would adjust the docs on `acceptField` to just say "Whether or not this field is of interest to the visitor". It shouldn't be term-specific. -- This is an automated message

[GitHub] [lucene] rmuir commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout

2023-09-05 Thread via GitHub
rmuir commented on PR #12535: URL: https://github.com/apache/lucene/pull/12535#issuecomment-1706783612 The 500ms connect timeout on the client: extremely suspicious (as far as jenkins failure). Make it at least 3 seconds or something? Doesn't matter if its being used by jenkins or not, ther

[GitHub] [lucene] uschindler commented on pull request #12535: LockVerifyServer does not need to reuse addresses nor set accept timeout

2023-09-05 Thread via GitHub
uschindler commented on PR #12535: URL: https://github.com/apache/lucene/pull/12535#issuecomment-1706816121 > The 500ms connect timeout on the client: extremely suspicious (as far as jenkins failure). Make it at least 3 seconds or something? Doesn't matter if its being used by jenkins or no

[GitHub] [lucene] Tony-X commented on issue #12536: Remove `lastPosBlockOffset` from term metadata for Lucene90PostingsFormat

2023-09-05 Thread via GitHub
Tony-X commented on issue #12536: URL: https://github.com/apache/lucene/issues/12536#issuecomment-1706979791 Today when it skips, the skipper can tell us 1) the offset of the position block we should seek to 2) how many positions it needs to skip within this block. This is because post-skip

[GitHub] [lucene] Tony-X commented on issue #12513: Try out a tantivy's term dictionary format

2023-09-05 Thread via GitHub
Tony-X commented on issue #12513: URL: https://github.com/apache/lucene/issues/12513#issuecomment-1707066423 > Do you know whether Tantivy is producing a truly minimal FST? Maybe @fulmicoton can shed more light on this topic :) A related question: can Tantivy read a Lucene-buil