Re: [PR] Lucene99HnswVectorsReader[.readFields] readability tweaks [lucene]

2024-07-12 Thread via GitHub
cpoerschke merged PR #13532: URL: https://github.com/apache/lucene/pull/13532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

Re: [PR] in KnnVectorsWriter reduce code duplication w.r.t. MergedVectorValues.merge(Float|Byte)VectorValues [lucene]

2024-07-12 Thread via GitHub
cpoerschke commented on code in PR #13539: URL: https://github.com/apache/lucene/pull/13539#discussion_r1675576429 ## lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java: ## @@ -22,6 +22,7 @@ import java.util.ArrayList; import java.util.Arrays; import java.uti

Re: [PR] in KnnVectorsWriter reduce code duplication w.r.t. MergedVectorValues.merge(Float|Byte)VectorValues [lucene]

2024-07-12 Thread via GitHub
cpoerschke merged PR #13539: URL: https://github.com/apache/lucene/pull/13539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

[I] "gradlew clean check" results in internal gradle error "Unable to make progress running work." [lucene]

2024-07-12 Thread via GitHub
dweiss opened a new issue, #13567: URL: https://github.com/apache/lucene/issues/13567 ### Description As reported by Chris, running "gradlew clean check" on main results in this nasty (or its variations): ``` Unable to make progress running work. The following items are queued f

Re: [I] "gradlew clean check" results in internal gradle error "Unable to make progress running work." [lucene]

2024-07-12 Thread via GitHub
dweiss commented on issue #13567: URL: https://github.com/apache/lucene/issues/13567#issuecomment-2225232211 I think this comment provides an explanation of what is happening: https://github.com/gradle/gradle/issues/23585#issuecomment-1403862031 -- This is an automated message f

Re: [PR] Minor cleanup in some Facet tests [lucene]

2024-07-12 Thread via GitHub
mikemccand commented on PR #13489: URL: https://github.com/apache/lucene/pull/13489#issuecomment-2225379585 Thank you @slow-J and @stefanvodita! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-07-12 Thread via GitHub
cpoerschke commented on code in PR #13525: URL: https://github.com/apache/lucene/pull/13525#discussion_r1675724584 ## lucene/core/src/java/org/apache/lucene/index/FieldInfo.java: ## @@ -92,6 +97,8 @@ public FieldInfo( int vectorDimension, VectorEncoding vectorEncod

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-07-12 Thread via GitHub
cpoerschke commented on code in PR #13525: URL: https://github.com/apache/lucene/pull/13525#discussion_r1675735652 ## lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatMultiVectorsWriter.java: ## @@ -0,0 +1,824 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-07-12 Thread via GitHub
cpoerschke commented on code in PR #13525: URL: https://github.com/apache/lucene/pull/13525#discussion_r1675737097 ## lucene/core/src/java/org/apache/lucene/index/IndexingChain.java: ## @@ -1527,15 +1549,20 @@ void setPoints(int dimensionCount, int indexDimensionCount, int numB

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-07-12 Thread via GitHub
cpoerschke commented on code in PR #13525: URL: https://github.com/apache/lucene/pull/13525#discussion_r1675739599 ## lucene/core/src/java/org/apache/lucene/index/FieldInfos.java: ## @@ -452,7 +465,8 @@ synchronized int addOrGet(FieldInfo fi) { new FieldVectorPr

[PR] Compute facets while collecting [lucene]

2024-07-12 Thread via GitHub
epotyom opened a new pull request, #13568: URL: https://github.com/apache/lucene/pull/13568 @Shradha26 and I are working on a new faceting implementation in the sandbox module. With this, we are proposing the following new features to Lucene’s faceting and aggregation capabilities -

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2225606218 I tested using KnnGraphTester and in the process changed this to handle multiple levels (it just seemed to make sense form a consistency perspective). I also found there were a couple o

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
benwtrent commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676046779 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -408,7 +410,23 @@ private void finish() throws IOException { } private void

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676077428 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -408,7 +410,23 @@ private void finish() throws IOException { } private void c

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676077871 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswUtil.java: ## @@ -30,46 +30,83 @@ import org.apache.lucene.index.IndexReader; import org.apache.lucene.index

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676091586 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -408,7 +410,23 @@ private void finish() throws IOException { } private void c

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
benwtrent commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676111453 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -408,7 +410,23 @@ private void finish() throws IOException { } private void

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676154393 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswUtil.java: ## @@ -30,46 +30,83 @@ import org.apache.lucene.index.IndexReader; import org.apache.lucene.index

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676160815 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswUtil.java: ## @@ -30,46 +30,83 @@ import org.apache.lucene.index.IndexReader; import org.apache.lucene.index

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on code in PR #13566: URL: https://github.com/apache/lucene/pull/13566#discussion_r1676192442 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswUtil.java: ## @@ -30,46 +30,83 @@ import org.apache.lucene.index.IndexReader; import org.apache.lucene.index

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2225970694 This version also applies the connectedness-checking and patching to concurrent graph build. It was a trivial addition. I tested and it seems to maybe help a bit: |test | r

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
benwtrent commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2226232962 I have benchmarked 2 data sets with 2 scenarios (int4 and int7 quantization) and have found no significant difference in runtime between this branch and the main branch. |

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-07-12 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2226299595 To make things look bad, I think we need to reduce M and/or beamWidth? In my test I saw some impact with M=16 and beamWidth=50 -- This is an automated message from the Apache Git Serv

Re: [I] Merge on Commit: No merges if new data is flushed (but not committed) [lucene]

2024-07-12 Thread via GitHub
ameyakarve commented on issue #13537: URL: https://github.com/apache/lucene/issues/13537#issuecomment-2226555771 Yeah this works on 9.9 Thanks for pointing it out. I'll close this issue -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] Merge on Commit: No merges if new data is flushed (but not committed) [lucene]

2024-07-12 Thread via GitHub
ameyakarve closed issue #13537: Merge on Commit: No merges if new data is flushed (but not committed) URL: https://github.com/apache/lucene/issues/13537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [Issue #13482] Fix english grammar error in Field that stores a per-document long values for scoring [lucene]

2024-07-12 Thread via GitHub
github-actions[bot] commented on PR #13490: URL: https://github.com/apache/lucene/pull/13490#issuecomment-2226563782 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi