Re: [PR] Udpate ReadTask to not rely on search(Query, Collector) [lucene]

2024-07-30 Thread via GitHub
javanna commented on code in PR #13602: URL: https://github.com/apache/lucene/pull/13602#discussion_r1696627499 ## lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/SearchWithCollectorTask.java: ## @@ -46,17 +46,17 @@ public boolean withCollector() { } @

Re: [PR] Udpate ReadTask to not rely on search(Query, Collector) [lucene]

2024-07-30 Thread via GitHub
javanna commented on code in PR #13602: URL: https://github.com/apache/lucene/pull/13602#discussion_r1696628573 ## lucene/benchmark/conf/collector.alg: ## @@ -17,11 +17,10 @@ # - # multi val p

Re: [PR] Introduce IndexSearcher#searchLeaf(LeafReaderContext, Weight, Collector) method [lucene]

2024-07-30 Thread via GitHub
javanna commented on PR #13603: URL: https://github.com/apache/lucene/pull/13603#issuecomment-2257899247 Thanks a lot for your review @gsmiller , much appreciated, and for the reminder around the CHANGES entry. Would you like to review the wording perhaps? Otherwise, this should be good to

Re: [I] Remove the @Deprecated methods from TopScoreDocCollector and TopFieldCollector [lucene]

2024-07-30 Thread via GitHub
javanna closed issue #13499: Remove the @Deprecated methods from TopScoreDocCollector and TopFieldCollector URL: https://github.com/apache/lucene/issues/13499 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Removing all deprecated TopScoreDocCollector + TopFieldCollector methods (#create, #createSharedManager) [lucene]

2024-07-30 Thread via GitHub
javanna merged PR #13617: URL: https://github.com/apache/lucene/pull/13617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [I] Remove the @Deprecated methods from TopScoreDocCollector and TopFieldCollector [lucene]

2024-07-30 Thread via GitHub
javanna closed issue #13499: Remove the @Deprecated methods from TopScoreDocCollector and TopFieldCollector URL: https://github.com/apache/lucene/issues/13499 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Removing all deprecated TopScoreDocCollector + TopFieldCollector methods (#create, #createSharedManager) [lucene]

2024-07-30 Thread via GitHub
javanna commented on PR #13617: URL: https://github.com/apache/lucene/pull/13617#issuecomment-2257918667 Thanks again @slow-J . Heads up, I moved the changes entry to the "API changes" section above after merging, I did not catch this while reviewing :) -- This is an automated message fro

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
epotyom commented on PR #13559: URL: https://github.com/apache/lucene/pull/13559#issuecomment-2258002016 I've added new `BitSet#firstSetBitInRange` method with maybe naive implementations for both `SparseFixedBitSet` and `FixedBitSet`. I've also changed `BlockJoinSelector` to use it. `lucen

Re: [PR] Removing all deprecated TopScoreDocCollector + TopFieldCollector methods (#create, #createSharedManager) [lucene]

2024-07-30 Thread via GitHub
slow-J commented on PR #13617: URL: https://github.com/apache/lucene/pull/13617#issuecomment-2258011155 My bad, sorry for the mistake, thanks for catching it and the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
jpountz commented on code in PR #13585: URL: https://github.com/apache/lucene/pull/13585#discussion_r1696774526 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -0,0 +1,2028 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
jpountz commented on code in PR #13585: URL: https://github.com/apache/lucene/pull/13585#discussion_r1696776133 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -0,0 +1,2028 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
jpountz commented on code in PR #13585: URL: https://github.com/apache/lucene/pull/13585#discussion_r1696777219 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -0,0 +1,2028 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
jpountz commented on code in PR #13585: URL: https://github.com/apache/lucene/pull/13585#discussion_r1696780288 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -0,0 +1,2028 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
jpountz commented on code in PR #13585: URL: https://github.com/apache/lucene/pull/13585#discussion_r1696779655 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/Lucene912PostingsReader.java: ## @@ -0,0 +1,2028 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
jpountz commented on PR #13585: URL: https://github.com/apache/lucene/pull/13585#issuecomment-2258120692 Thanks @mikemccand for taking a look at this large PR! I think I applied all your suggestions. The format docs should be up-to-date wrt how skip data is stored, and I did the codec dance

Re: [PR] Release memory for cancelled tasks earlier in TaskExecutor [lucene]

2024-07-30 Thread via GitHub
javanna commented on PR #13609: URL: https://github.com/apache/lucene/pull/13609#issuecomment-2258193070 Ok , my hope was that we can somehow test cancellations with a scenario that's closer to reality, but I do see challenges. I also spent quite a bit of time on this and hit different issu

Re: [PR] Release memory for cancelled tasks earlier in TaskExecutor [lucene]

2024-07-30 Thread via GitHub
original-brownbear commented on PR #13609: URL: https://github.com/apache/lucene/pull/13609#issuecomment-2258283776 @javanna > Perhaps we could at least write a test that fails without the change you made, around nulling memory? I'm really sorry. I thought about this quite a b

Re: [PR] Deduplicate bytes for `FieldReader#rootCode` [lucene]

2024-07-30 Thread via GitHub
jpountz commented on code in PR #13610: URL: https://github.com/apache/lucene/pull/13610#discussion_r1696951086 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/blocktree/FieldReader.java: ## @@ -100,6 +99,14 @@ public final class FieldReader extends Terms { w.clo

Re: [PR] Introduce IndexSearcher#searchLeaf(LeafReaderContext, Weight, Collector) method [lucene]

2024-07-30 Thread via GitHub
javanna merged PR #13603: URL: https://github.com/apache/lucene/pull/13603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Introduce IndexSearcher#searchLeaf(LeafReaderContext, Weight, Collector) method [lucene]

2024-07-30 Thread via GitHub
javanna commented on PR #13603: URL: https://github.com/apache/lucene/pull/13603#issuecomment-2258605434 Thank you again for the review @gsmiller ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] [9.x] Introduce IndexSearcher#searchLeaf(LeafReaderContext, Weight, Collector) method [lucene]

2024-07-30 Thread via GitHub
javanna opened a new pull request, #13619: URL: https://github.com/apache/lucene/pull/13619 Backport of #13603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Introduce IndexSearcher#searchLeaf(LeafReaderContext, Weight, Collector) method [lucene]

2024-07-30 Thread via GitHub
gsmiller commented on PR #13603: URL: https://github.com/apache/lucene/pull/13603#issuecomment-2258647048 @javanna yeah no problem. All looks good to me. +1 to backporting to 9.12. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [9.x] Introduce IndexSearcher#searchLeaf(LeafReaderContext, Weight, Collector) method [lucene]

2024-07-30 Thread via GitHub
javanna merged PR #13619: URL: https://github.com/apache/lucene/pull/13619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Inline skip data into postings lists [lucene]

2024-07-30 Thread via GitHub
mikemccand commented on PR #13585: URL: https://github.com/apache/lucene/pull/13585#issuecomment-2258788196 > Note that I removed `TestLazyProxSkipping`, which assumed separate skip data and postings. YAY! > I plan on merging soon, as this PR is now in a state where conflicts

Re: [I] TestFSTs.testRandomWords reproducible NPE [lucene]

2024-07-30 Thread via GitHub
slow-J commented on issue #13174: URL: https://github.com/apache/lucene/issues/13174#issuecomment-2258837473 Should we close this if this issue no longer manifests? I cannot repro on `main` at the newest commit https://github.com/apache/lucene/commit/30c965ea575a5e75c2bf724a340aa690d82f1ec5

Re: [PR] Add support for intra-segment search concurrency [lucene]

2024-07-30 Thread via GitHub
javanna commented on code in PR #13542: URL: https://github.com/apache/lucene/pull/13542#discussion_r1697317194 ## lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java: ## @@ -328,42 +336,65 @@ protected LeafSlice[] slices(List leaves) { /** Static method to segr

Re: [PR] Removing all deprecated TopScoreDocCollector + TopFieldCollector methods (#create, #createSharedManager) [lucene]

2024-07-30 Thread via GitHub
javanna commented on PR #13617: URL: https://github.com/apache/lucene/pull/13617#issuecomment-2258904454 > My bad, sorry for the mistake, thanks for catching it and the review! no big deal, just a small oversight. Cheers! -- This is an automated message from the Apache Git Service.

Re: [PR] Add timeout support to AbstractVectorSimilarityQuery [lucene]

2024-07-30 Thread via GitHub
kaivalnp commented on code in PR #13285: URL: https://github.com/apache/lucene/pull/13285#discussion_r1697413263 ## lucene/core/src/java/org/apache/lucene/search/AbstractVectorSimilarityQuery.java: ## @@ -103,16 +114,18 @@ public Explanation explain(LeafReaderContext context, in

Re: [PR] Add timeout support to AbstractVectorSimilarityQuery [lucene]

2024-07-30 Thread via GitHub
kaivalnp commented on code in PR #13285: URL: https://github.com/apache/lucene/pull/13285#discussion_r1697415078 ## lucene/core/src/java/org/apache/lucene/search/AbstractVectorSimilarityQuery.java: ## @@ -143,27 +156,23 @@ protected boolean match(int doc) { }

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
gsmiller commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697210371 ## lucene/core/src/java/org/apache/lucene/util/BitSet.java: ## @@ -92,6 +92,12 @@ public void clear() { */ public abstract int nextSetBit(int index); + /**

Re: [I] TestFSTs.testRandomWords reproducible NPE [lucene]

2024-07-30 Thread via GitHub
stefanvodita commented on issue #13174: URL: https://github.com/apache/lucene/issues/13174#issuecomment-2259007249 It's likely that an issue might no longer be reproducible on a more recent commit because an unrelated change would have impacted the random numbers we generate. I wouldn't ass

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-07-30 Thread via GitHub
rmuir commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1697452900 ## lucene/core/src/c/dotProduct.c: ## @@ -0,0 +1,143 @@ +// dotProduct.c + +#include +#include + +#ifdef __ARM_ACLE +#include +#endif + +#if (defined(__ARM_FEATURE_SV

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-07-30 Thread via GitHub
rmuir commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1697455240 ## lucene/core/src/c/dotProduct.h: ## @@ -0,0 +1,4 @@ + +int32_t vdot8s_sve(int8_t* vec1[], int8_t* vec2, int32_t limit); +int32_t vdot8s_neon(int8_t* vec1[], int8_t* ve

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-07-30 Thread via GitHub
rmuir commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1697462600 ## lucene/core/src/c/dotProduct.c: ## @@ -0,0 +1,143 @@ +// dotProduct.c + +#include +#include + +#ifdef __ARM_ACLE +#include +#endif + +#if (defined(__ARM_FEATURE_SV

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-07-30 Thread via GitHub
rmuir commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1697469288 ## lucene/core/src/c/dotProduct.c: ## @@ -0,0 +1,143 @@ +// dotProduct.c + +#include +#include + +#ifdef __ARM_ACLE +#include +#endif + +#if (defined(__ARM_FEATURE_SV

[I] Reproducible unit test failure [lucene]

2024-07-30 Thread via GitHub
slow-J opened a new issue, #13620: URL: https://github.com/apache/lucene/issues/13620 ### Description https://jenkins.thetaphi.de/job/Lucene-main-MacOSX/11653/ I can repro with `./gradlew test --tests TestTopDocsCollector.testResultsOrder -Dtests.seed=207A6071B3338CA6` T

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-07-30 Thread via GitHub
rmuir commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1697474113 ## lucene/core/src/c/dotProduct.c: ## @@ -0,0 +1,143 @@ +// dotProduct.c + +#include +#include + +#ifdef __ARM_ACLE +#include +#endif + +#if (defined(__ARM_FEATURE_SV

[PR] Fix failing unit test - TestTopDocsCollector#testResultsOrder [lucene]

2024-07-30 Thread via GitHub
slow-J opened a new pull request, #13621: URL: https://github.com/apache/lucene/pull/13621 Closes #13620 I believe this bug happens due to having 2 collectors with the same indexes and the score of `9.17561f` is never inserted into the pq, failing the assertion. Another potent

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
epotyom commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697607067 ## lucene/core/src/java/org/apache/lucene/util/BitSet.java: ## @@ -92,6 +92,12 @@ public void clear() { */ public abstract int nextSetBit(int index); + /** +

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
epotyom commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697608653 ## lucene/join/src/java/org/apache/lucene/search/join/BlockJoinSelector.java: ## @@ -64,14 +64,19 @@ public boolean get(int docID) { return false;

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
epotyom commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697615867 ## lucene/core/src/java/org/apache/lucene/util/FixedBitSet.java: ## @@ -291,6 +291,32 @@ public int nextSetBit(int index) { return DocIdSetIterator.NO_MORE_DOCS;

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
gsmiller commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697623524 ## lucene/core/src/java/org/apache/lucene/util/FixedBitSet.java: ## @@ -291,6 +291,32 @@ public int nextSetBit(int index) { return DocIdSetIterator.NO_MORE_DOCS;

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
epotyom commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697616026 ## lucene/core/src/java/org/apache/lucene/util/SparseFixedBitSet.java: ## @@ -353,14 +355,47 @@ public int nextSetBit(int i) { final long indexBits = index >>> i6

Re: [PR] SparseFixedBitSet#firstDoc: reduce number of `indices` iterations for a bit set that is not fully built yet. [lucene]

2024-07-30 Thread via GitHub
gsmiller commented on code in PR #13559: URL: https://github.com/apache/lucene/pull/13559#discussion_r1697630535 ## lucene/core/src/java/org/apache/lucene/util/BitSet.java: ## @@ -92,6 +92,12 @@ public void clear() { */ public abstract int nextSetBit(int index); + /**

Re: [PR] WIP do not merge [lucene]

2024-07-30 Thread via GitHub
github-actions[bot] commented on PR #13577: URL: https://github.com/apache/lucene/pull/13577#issuecomment-2259403530 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [I] KnnFloatVectorQuery misses highest-ranking results that FloatVectorSimilarityQuery retrieves [lucene]

2024-07-30 Thread via GitHub
david-sitsky closed issue #13611: KnnFloatVectorQuery misses highest-ranking results that FloatVectorSimilarityQuery retrieves URL: https://github.com/apache/lucene/issues/13611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and