Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-06-05 Thread via GitHub
benwtrent commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2149954395 @navneet1v > Does this mean now Lucene supports BitVectorsFormat officially? The answer is no. > Or it was more a prototype and not intended for production use?

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-06-04 Thread via GitHub
navneet1v commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2148642670 @benwtrent I am little confused here. I am still looking for an ans of this question: `Does this mean now Lucene supports BitVectorsFormat officially? Or it was more a prototyp

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-05-29 Thread via GitHub
benwtrent commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2137382358 1. No, BitVector format is not in the backwards compatible package. 2. Correct, there have been previous discussions in an effort to add it as a similarity value, but those conversat

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-05-25 Thread via GitHub
navneet1v commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2132065571 @benwtrent I see that with this PR and enabled the flat vectors format easier to extend. You showed it with an example for BitVectorsFormat. 1. Does this mean now Lucene supports Bi

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-17 Thread via GitHub
benwtrent merged PR #13288: URL: https://github.com/apache/lucene/pull/13288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.a

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-16 Thread via GitHub
jimczi commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2059552949 > I am not a huge fan of how every scorer can now just get a "queryOrdinal" and overwrite whatever query was passed to it. Yep that's tricky. I couldn't find a better way since my g

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-15 Thread via GitHub
benwtrent commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2057715372 @jimczi OK, I read a bit more of your suggestion. I am not a huge fan of how every scorer can now just get a "queryOrdinal" and overwrite whatever query was passed to it.

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-15 Thread via GitHub
ChrisHegarty commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2057682888 I would like to suggest that we reintroduce `getSlice`. The `getSlice` method is critical to any serious implementation that wants to take things into its own hands. The `getSlice`

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-12 Thread via GitHub
uschindler commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2052630571 > Hey @uschindler I didn't want to move forward on merging without your thoughts. This is a separate idea from: #13200 Will check tomorrow. -- This is an automated message fr

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-12 Thread via GitHub
benwtrent commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2052231527 @jimczi I do like the further simplification. I can see about pulling in some of your ideas -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-12 Thread via GitHub
benwtrent commented on PR #13288: URL: https://github.com/apache/lucene/pull/13288#issuecomment-2052087989 Hey @uschindler I didn't want to move forward on merging without your thoughts. This is a separate idea from: https://github.com/apache/lucene/pull/13200 This change is more inl

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-12 Thread via GitHub
benwtrent commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1562457295 ## lucene/codecs/src/java/org/apache/lucene/codecs/simpletext/SimpleTextKnnVectorsReader.java: ## @@ -472,9 +472,8 @@ private void readVector(byte[] value) throws IO

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-12 Thread via GitHub
tteofili commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1562353324 ## lucene/core/src/java/org/apache/lucene/codecs/hnsw/package-info.java: ## @@ -0,0 +1,19 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-12 Thread via GitHub
jpountz commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1562108834 ## lucene/core/src/java/org/apache/lucene/codecs/hnsw/FlatBitVectorsScorer.java: ## @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-11 Thread via GitHub
jpountz commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1562101541 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/bitvectors/HnswBitVectorsFormat.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-11 Thread via GitHub
jimczi commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1561065045 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/bitvectors/HnswBitVectorsFormat.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-11 Thread via GitHub
benwtrent commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1560987003 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/bitvectors/HnswBitVectorsFormat.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-11 Thread via GitHub
jimczi commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1560956676 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/bitvectors/HnswBitVectorsFormat.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-11 Thread via GitHub
benwtrent commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1560939497 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/bitvectors/HnswBitVectorsFormat.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-10 Thread via GitHub
jimczi commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1560095226 ## lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatVectorsWriter.java: ## @@ -71,7 +72,9 @@ public final class Lucene99FlatVectorsWriter extends Fla

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-10 Thread via GitHub
ChrisHegarty commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1559885835 ## lucene/analysis/common/src/java/org/apache/lucene/analysis/synonym/word2vec/Word2VecModel.java: ## @@ -29,7 +29,7 @@ * * @lucene.experimental */ -public

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-10 Thread via GitHub
benwtrent commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1559873050 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/codecs/bitvectors/HnswBitVectorsFormat.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-10 Thread via GitHub
benwtrent commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1559871922 ## lucene/core/src/java/org/apache/lucene/codecs/lucene95/OffHeapFloatVectorValues.java: ## @@ -79,7 +84,7 @@ public static OffHeapFloatVectorValues load( ret

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-10 Thread via GitHub
benwtrent commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1559871450 ## lucene/core/src/java/org/apache/lucene/codecs/FlatVectorsScorer.java: ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mo

Re: [PR] Add BitVectors format and make flat vectors format easier to extend [lucene]

2024-04-10 Thread via GitHub
jpountz commented on code in PR #13288: URL: https://github.com/apache/lucene/pull/13288#discussion_r1559759260 ## lucene/core/src/java/org/apache/lucene/codecs/lucene95/OffHeapFloatVectorValues.java: ## @@ -79,7 +84,7 @@ public static OffHeapFloatVectorValues load( retur