Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-11-04 Thread via GitHub
goankur commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1828574298 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -291,25 +296,125 @@ private float squareDistanceBody(float[] a,

Re: [PR] Add XNOR in FixedBitSet. [lucene]

2024-11-04 Thread via GitHub
vsop-479 closed pull request #13540: Add XNOR in FixedBitSet. URL: https://github.com/apache/lucene/pull/13540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-11-04 Thread via GitHub
goankur commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1828579554 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -291,25 +296,125 @@ private float squareDistanceBody(float[] a,

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-11-04 Thread via GitHub
goankur commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1828577276 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -291,25 +296,125 @@ private float squareDistanceBody(float[] a,

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-11-04 Thread via GitHub
goankur commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1828576127 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -291,25 +296,125 @@ private float squareDistanceBody(float[] a,

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-11-04 Thread via GitHub
goankur commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1828574298 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -291,25 +296,125 @@ private float squareDistanceBody(float[] a,

Re: [PR] Speedup PriorityQueue a little [lucene]

2024-11-04 Thread via GitHub
github-actions[bot] commented on PR #13936: URL: https://github.com/apache/lucene/pull/13936#issuecomment-2455970248 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] A few small tweaks to VectorUtil#findNextGEQ [lucene]

2024-11-04 Thread via GitHub
gsmiller merged PR #13972: URL: https://github.com/apache/lucene/pull/13972 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.ap

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-11-04 Thread via GitHub
vigyasharma commented on PR #13525: URL: https://github.com/apache/lucene/pull/13525#issuecomment-2455892813 One use-case for multi-vectors is indexing product aspects as separate embeddings for e-commerce search. At Amazon Product Search (where I work), we'd like to experiment with separat

Re: [PR] New JMH benchmark method - vdot8s that implement int8 dotProduct in C… [lucene]

2024-11-04 Thread via GitHub
goankur commented on code in PR #13572: URL: https://github.com/apache/lucene/pull/13572#discussion_r1828364953 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -291,25 +296,125 @@ private float squareDistanceBody(float[] a,

Re: [PR] Multireader Support in Searcher Manager [lucene]

2024-11-04 Thread via GitHub
mikemccand commented on PR #13976: URL: https://github.com/apache/lucene/pull/13976#issuecomment-2455571526 Thanks @Shibi-bala -- I agree it's odd it was scoped to just `DirectoryReader` -- any `IndexReader` should work as long as it can `openIfChanged` on itself. I think `English.ja

Re: [PR] Revert "Disjunction as CompetitiveIterator for numeric dynamic pruning (#13221)" [lucene]

2024-11-04 Thread via GitHub
javanna commented on code in PR #13971: URL: https://github.com/apache/lucene/pull/13971#discussion_r1828222514 ## lucene/core/src/java/org/apache/lucene/util/IntArrayDocIdSet.java: ## @@ -34,23 +34,15 @@ public final class IntArrayDocIdSet extends DocIdSet { private final in

[PR] Remove `supportsConcurrency` flag from top-docs collector managers. [lucene]

2024-11-04 Thread via GitHub
jpountz opened a new pull request, #13977: URL: https://github.com/apache/lucene/pull/13977 Our collector managers have a `supportsConcurrency` flag to optimize the case when they are used in a single thread. This PR proposes to remove this flag now that the optimization doesn't do much as

Re: [PR] Remove `supportsConcurrency` flag from top-docs collector managers. [lucene]

2024-11-04 Thread via GitHub
jpountz commented on PR #13977: URL: https://github.com/apache/lucene/pull/13977#issuecomment-2455325803 Here is the output of luceneutil on wikibigall with `-searchConcurrency 0`. Despite the removal of optimizations for the case when there is no search concurrency, all variations look lik

[PR] Multireader Support in Searcher Manager [lucene]

2024-11-04 Thread via GitHub
Shibi-bala opened a new pull request, #13976: URL: https://github.com/apache/lucene/pull/13976 ### Description Copied from https://github.com/apache/lucene/issues/13975 I'd like to use MultiReader inside my searcher manager, but currently there is only support for DirectoryRead

[I] Searcher Manger with support for Multi-Reader [lucene]

2024-11-04 Thread via GitHub
Shibi-bala opened a new issue, #13975: URL: https://github.com/apache/lucene/issues/13975 ### Description I'd like to use MultiReader inside my searcher manager, but currently there is only support for DirectoryReader. Not sure about the context for. why this was the case initially,

Re: [PR] Add new Directory implementation for AWS S3 [lucene]

2024-11-04 Thread via GitHub
reta commented on PR #13949: URL: https://github.com/apache/lucene/pull/13949#issuecomment-2454758505 Just to share similar developments, we did S3 filesystem plugin store prototype for OpenSearch based on the same library [1], it works but the latency wise, it is not great (obviously, no c

Re: [PR] Add new Directory implementation for AWS S3 [lucene]

2024-11-04 Thread via GitHub
albogdano commented on PR #13949: URL: https://github.com/apache/lucene/pull/13949#issuecomment-2454576266 @rmuir @josefschiefer27 I haven't yet tried experimenting with S3 fuse mounts but I doubt the performance will be any better than using the S3 API directly. Also, there are some specif

Re: [I] Unable to Tessellate shape for a valid Polygon according to GDAL/OGR and PostGIS [lucene]

2024-11-04 Thread via GitHub
garaud commented on issue #13841: URL: https://github.com/apache/lucene/issues/13841#issuecomment-2454260757 Thank you for your answers from both of you. It looks like a tricky issue :) I'll keep on eye on this thread. -- This is an automated message from the Apache Git Servi

Re: [PR] Revert "Disjunction as CompetitiveIterator for numeric dynamic pruning (#13221)" [lucene]

2024-11-04 Thread via GitHub
jpountz commented on code in PR #13971: URL: https://github.com/apache/lucene/pull/13971#discussion_r1827427614 ## lucene/core/src/java/org/apache/lucene/util/IntArrayDocIdSet.java: ## @@ -34,23 +34,15 @@ public final class IntArrayDocIdSet extends DocIdSet { private final in

Re: [PR] Move postings back to int[] to take advantage of having more lanes per vector. [lucene]

2024-11-04 Thread via GitHub
jpountz commented on PR #13968: URL: https://github.com/apache/lucene/pull/13968#issuecomment-2454071806 Nightly benchmarks confirmed a speedup from the combination of #13958 (SIMD for advancing within a block) and this PR. ([CountAndHighHigh](https://benchmarks.mikemccandless.com/CountAndH

Re: [PR] A few small tweaks to VectorUtil#findNextGEQ [lucene]

2024-11-04 Thread via GitHub
jpountz commented on code in PR #13972: URL: https://github.com/apache/lucene/pull/13972#discussion_r1827331927 ## lucene/core/src/java/org/apache/lucene/codecs/lucene101/Lucene101PostingsReader.java: ## @@ -601,7 +599,7 @@ public int advance(int target) throws IOException {