Re: [I] Refactor HNSW graph build such that concurrent build won't impact single thread build [lucene]

2023-10-28 Thread via GitHub
vigyasharma commented on issue #12732: URL: https://github.com/apache/lucene/issues/12732#issuecomment-1783928942 I'm trying to familiarize myself more with Lucene's HNSW implementation, and would like to help with this task. -- This is an automated message from the Apache Git Service. To

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
uschindler commented on code in PR #12731: URL: https://github.com/apache/lucene/pull/12731#discussion_r1375324223 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -77,6 +77,47 @@ final class PanamaVectorUtilSupport implement

Re: [PR] [DRAFT] Add unsigned byte vector operations for uint8 quantization [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on code in PR #12694: URL: https://github.com/apache/lucene/pull/12694#discussion_r1375318798 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -352,6 +382,11 @@ private int dotProductBody512(byte[] a, b

Re: [PR] Ensure negative scores are not returned by vector similarity functions [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on code in PR #12727: URL: https://github.com/apache/lucene/pull/12727#discussion_r1375318432 ## lucene/core/src/java/org/apache/lucene/index/VectorSimilarityFunction.java: ## @@ -70,7 +74,11 @@ public float compare(byte[] v1, byte[] v2) { COSINE {

Re: [PR] Ensure negative scores are not returned by vector similarity functions [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on code in PR #12727: URL: https://github.com/apache/lucene/pull/12727#discussion_r1375316778 ## lucene/core/src/java/org/apache/lucene/index/VectorSimilarityFunction.java: ## @@ -70,7 +74,11 @@ public float compare(byte[] v1, byte[] v2) { COSINE {

Re: [PR] Ensure negative scores are not returned by vector similarity functions [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on PR #12727: URL: https://github.com/apache/lucene/pull/12727#issuecomment-1783913908 Hi @benwtrent I think that this is fine - LGTM, just dropping a few small comments / questions. I grabbed and modified your test, and was able to repo this on both my Linu

Re: [PR] Concurrent HNSW Merge [lucene]

2023-10-28 Thread via GitHub
zhaih merged PR #12660: URL: https://github.com/apache/lucene/pull/12660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apach

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on PR #12731: URL: https://github.com/apache/lucene/pull/12731#issuecomment-1783869625 .. and yes (I've not forgotten), we need something like a `java.lang.Architecture/Platform`, that is queryable for such low-level support (rather than resorting to beans - which act

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on PR #12731: URL: https://github.com/apache/lucene/pull/12731#issuecomment-1783869078 ha! So just removing the overly aggressive unrolling in cosine improves things. The check on FMA is nice - I had similar thoughts ( you just beat me to it! ), and it inlines nicel

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
ChrisHegarty commented on code in PR #12731: URL: https://github.com/apache/lucene/pull/12731#discussion_r1375278303 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -77,6 +77,47 @@ final class PanamaVectorUtilSupport impleme

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
uschindler commented on code in PR #12731: URL: https://github.com/apache/lucene/pull/12731#discussion_r1375252807 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -77,6 +77,47 @@ final class PanamaVectorUtilSupport implement

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
rmuir commented on code in PR #12731: URL: https://github.com/apache/lucene/pull/12731#discussion_r1375248973 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -77,6 +77,47 @@ final class PanamaVectorUtilSupport implements Ve

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
rmuir commented on code in PR #12731: URL: https://github.com/apache/lucene/pull/12731#discussion_r1375244023 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -77,6 +77,47 @@ final class PanamaVectorUtilSupport implements Ve

Re: [PR] Speedup float cosine vectors, use FMA where fast and available to reduce error [lucene]

2023-10-28 Thread via GitHub
uschindler commented on code in PR #12731: URL: https://github.com/apache/lucene/pull/12731#discussion_r1375243087 ## lucene/core/src/java20/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java: ## @@ -77,6 +77,47 @@ final class PanamaVectorUtilSupport implement

Re: [PR] Fix test failures for TestCoreParser#testSpanNearQueryWithoutSlopXML [lucene]

2023-10-28 Thread via GitHub
uschindler commented on PR #12724: URL: https://github.com/apache/lucene/pull/12724#issuecomment-1783746086 Hi. No more brnaches needed. The older branches do not need it as the issue in idk was already fixed, so error no longer happens (was only in build 20 of jdk-22). P.S.: y