Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-31 Thread via GitHub
mikemccand commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2450133029 > Could you add a CHANGES entry in 9.12 for your bug fix for 9.12.1? Ahh yes sorry I will do that today! -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-23 Thread via GitHub
benwtrent commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2432746347 @mikemccand I have a PR open for this bug fix for 9.12. Will merge soon. Could you add a CHANGES entry in 9.12 for your bug fix for 9.12.1? -- This is an automated message fro

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-23 Thread via GitHub
mikemccand commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2432474472 > This seems like something we maybe ought to make the user community aware of. +1 thanks @msokolov. > @msokolov could we do a simpler patch for 9.12.1? +1. 9.12.

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-23 Thread via GitHub
mikemccand commented on code in PR #13910: URL: https://github.com/apache/lucene/pull/13910#discussion_r1812937509 ## lucene/backward-codecs/src/java/org/apache/lucene/backward_codecs/lucene90/Lucene90HnswVectorsReader.java: ## @@ -260,7 +260,7 @@ public void search(String field

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-22 Thread via GitHub
msokolov commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2429841476 There is another upgrade path -- if you started with 9.0 and then "upgraded" your index by rewriting it (eg with IndexUpdater tool) via merge to 9.1-9.7 you could subsequently read the

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-22 Thread via GitHub
msokolov commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2429836870 Yes, maybe we should -- I think it would be a one-liner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-22 Thread via GitHub
benwtrent commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2429748958 @msokolov could we do a simpler patch for 9.12.1? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-22 Thread via GitHub
msokolov commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2429428988 ok something like this: Dear Lucene user community, We recently uncovered a backwards compatibility bug that affects indexes created with version 9.0 containing KNN vector

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-21 Thread via GitHub
msokolov commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2427329494 > Since Lucene90 didn't support sparse vector values, I am not sure this is strictly necessary. But I can understand it from a consistency standpoint. After reflection, I don't th

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-17 Thread via GitHub
msokolov merged PR #13910: URL: https://github.com/apache/lucene/pull/13910 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.ap

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-17 Thread via GitHub
msokolov commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2419873679 @benwtrent I think I addressed all your comments except adding binary vectors. I think as long as the vectors are not too degenerate and always the same the test purpose is satisfied, b

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-16 Thread via GitHub
benwtrent commented on code in PR #13910: URL: https://github.com/apache/lucene/pull/13910#discussion_r1803132931 ## lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java: ## @@ -1906,4 +1916,122 @@ public void testMismatchedFields() thro

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-16 Thread via GitHub
benwtrent commented on code in PR #13910: URL: https://github.com/apache/lucene/pull/13910#discussion_r1803130185 ## lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java: ## @@ -1906,4 +1916,122 @@ public void testMismatchedFields() thro

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-16 Thread via GitHub
msokolov commented on code in PR #13910: URL: https://github.com/apache/lucene/pull/13910#discussion_r1803114051 ## lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java: ## @@ -1906,4 +1916,122 @@ public void testMismatchedFields() throw

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-16 Thread via GitHub
benwtrent commented on code in PR #13910: URL: https://github.com/apache/lucene/pull/13910#discussion_r1803081052 ## lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseKnnVectorsFormatTestCase.java: ## @@ -1906,4 +1916,122 @@ public void testMismatchedFields() thro

Re: [PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-14 Thread via GitHub
msokolov commented on PR #13910: URL: https://github.com/apache/lucene/pull/13910#issuecomment-2412421807 I wonder whether we should backport the fixes to the `Lucene90HnswVectorsReader`? I tend to think we ought to, although the usage might be tiny to nonexistent -- This is an automated

[PR] Add BaseKnnVectorsFormatTestCase.testRecall() and fix old codecs [lucene]

2024-10-14 Thread via GitHub
msokolov opened a new pull request, #13910: URL: https://github.com/apache/lucene/pull/13910 While exploring some recall-related failures in another PR I went looking for a unit test that checks HNSW/KNN recall and couldn't find any. I think we used to have one but maybe we removed it becau