Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1710505009 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
rmuir commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1710505391 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
rmuir commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2276947096 > This class in the Panama codebase is huge and hard to read, so splitting into byte/float implementations looks like a good cleanup. No public API is involved. Cool idea, thanks fo

Re: [I] HnwsGraph creates disconnected components [lucene]

2024-08-08 Thread via GitHub
msokolov closed issue #12627: HnwsGraph creates disconnected components URL: https://github.com/apache/lucene/issues/12627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] HnwsGraph creates disconnected components [lucene]

2024-08-08 Thread via GitHub
msokolov commented on issue #12627: URL: https://github.com/apache/lucene/issues/12627#issuecomment-2276936647 also backported to 9x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Made the UnifiedHighlighter's hasUnrecognizedQuery function processes FunctionQuery the same way as MatchAllDocsQuery and MatchNoDocsQuery queries for performance reasons. [lucene]

2024-08-08 Thread via GitHub
github-actions[bot] commented on PR #13165: URL: https://github.com/apache/lucene/pull/13165#issuecomment-2276934277 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] [WIP] Multi-Vector support for HNSW search [lucene]

2024-08-08 Thread via GitHub
github-actions[bot] commented on PR #13525: URL: https://github.com/apache/lucene/pull/13525#issuecomment-2276933809 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710374307 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/taxonomy/TaxonomyChildrenOrdinalIterator.java: ## @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710371930 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/ComparableUtils.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710365017 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/FacetFieldLeafCollector.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710360090 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/FacetFieldLeafCollector.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710358112 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/recorders/CountFacetRecorder.java: ## @@ -0,0 +1,189 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710346627 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/abstracts/FacetRecorder.java: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710344889 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/abstracts/FacetRecorder.java: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710343109 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/abstracts/OrdinalIterator.java: ## @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710338323 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/FacetLabel.java: ## @@ -178,6 +178,11 @@ public FacetLabel subpath(final int length) { } } + /** G

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710320207 ## lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java: ## @@ -45,58 +45,26 @@ class DrillSidewaysQuery extends Query { final Query baseQuery;

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710320207 ## lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java: ## @@ -45,58 +45,26 @@ class DrillSidewaysQuery extends Query { final Query baseQuery;

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710309544 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/LongRangeFacetCutter.java: ## @@ -0,0 +1,354 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710300104 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/LongValueFacetCutter.java: ## @@ -0,0 +1,173 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710295193 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/LongValueFacetCutter.java: ## @@ -0,0 +1,173 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710265556 ## lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java: ## @@ -349,45 +344,142 @@ private DrillDownQuery getDrillDownQuery( public ConcurrentDrillSide

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1710246000 ## lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java: ## @@ -349,45 +344,142 @@ private DrillDownQuery getDrillDownQuery( public ConcurrentDrillSide

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
msokolov merged PR #13566: URL: https://github.com/apache/lucene/pull/13566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.ap

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2276439009 I ran one more test using Cohere (768 dim) vectors. Looks OK to me, I'll merge this; at least it does no harm. test case | recall | t| M | beam | index time | connectivity

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709992304 ## lucene/CHANGES.txt: ## @@ -133,6 +142,8 @@ New Features DocValuesSkipper abstraction. A new flag is added to FieldType.java that configures whether to create

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709975382 ## lucene/core/src/java/org/apache/lucene/search/CollectorOwner.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709965505 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -37,15 +44,23 @@ private static T doPrivileged(Privilege

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709961906 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709952530 ## lucene/sandbox/src/java/module-info.java: ## @@ -19,13 +19,21 @@ module org.apache.lucene.sandbox { requires org.apache.lucene.core; requires org.apache.luce

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709932857 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/DoubleRangeFacetCutter.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709932857 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/DoubleRangeFacetCutter.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foun

Re: [I] `gradlew eclipse` no longer works [lucene]

2024-08-08 Thread via GitHub
uschindler commented on issue #13638: URL: https://github.com/apache/lucene/issues/13638#issuecomment-2276279148 Looks like this is caused by #13484 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[I] `gradlew eclipse` no longer works [lucene]

2024-08-08 Thread via GitHub
uschindler opened a new issue, #13638: URL: https://github.com/apache/lucene/issues/13638 ### Description Hi @dweiss, since your last cleanup, when trying to run `gradlew eclipse` it complains because the task does not exist. Looks ike the reason is that the root project no longer

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
benwtrent commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2276247855 > or maybe you're suggesting we change the way we search, not specifically something about this patch? That does kind of make sense to me - I'm not sure why the HNSW paper settled on a

Re: [I] Deprecate `COSINE` before Lucene 10 release [lucene]

2024-08-08 Thread via GitHub
benwtrent commented on issue #13281: URL: https://github.com/apache/lucene/issues/13281#issuecomment-2276245454 agreed, I think we will miss the window -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709835247 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/LongRangeNode.java: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709899394 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709866139 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [I] Deprecate `COSINE` before Lucene 10 release [lucene]

2024-08-08 Thread via GitHub
jpountz commented on issue #13281: URL: https://github.com/apache/lucene/issues/13281#issuecomment-2276232929 @benwtrent Shall we close this issue as "won't fix"? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709853131 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/OverlappingLongRangeFacetCutter.java: ## @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Sof

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2276225956 > What do we think of increasing the number of entry points to the lower level? or maybe you're suggesting we change the way we search, not specifically something about this patc

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709880508 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709770635 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/LongRangeFacetCutter.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2276098716 >>What do we think of increasing the number of entry points to the lower level? > My first attempt searched for 2 instead of 1, and reconnected both of them ... Oh wait I

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709749808 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/ForUtil.java: ## @@ -630,55 +637,55 @@ private static void decode1(DataInput in, long[] tmp, long[] longs)

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2276082590 > What do we think of increasing the number of entry points to the lower level? My first attempt searched for 2 instead of 1, and reconnected both of them, but doing this led to

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709713833 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/LongRangeFacetCutter.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709705545 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709691392 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/LongRangeFacetCutter.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
stefanvodita commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709557365 ## lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java: ## @@ -45,58 +45,26 @@ class DrillSidewaysQuery extends Query { final Query baseQue

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709520659 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
mikemccand commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1698434531 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/LongValueFacetCutter.java: ## @@ -0,0 +1,173 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709403728 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
benwtrent commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2275704679 > So this patch tries to ensure that every node on each level is reachable from one of the entry points of that level (which are just the nodes on the next higher level). This is easy

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709378126 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709362066 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709359720 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709358369 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] gh-12627: HnswGraphBuilder connects disconnected HNSW graph components [lucene]

2024-08-08 Thread via GitHub
msokolov commented on PR #13566: URL: https://github.com/apache/lucene/pull/13566#issuecomment-2275670617 I posted an updated patch that uses an approach that is more aligned with how we search. The idea is we don't need to care about strongly-connected graphs; we just want to make sure the

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709340084 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal.ve

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709339018 ## lucene/core/src/java/org/apache/lucene/codecs/lucene912/PostingIndexInput.java: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709336210 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -79,4 +113,15 @@ public VectorUtilSupport getVectorUtilSuppo

Re: [I] Lucene99FlatVectorsReader.getFloatVectorValues(): NPE: Cannot read field "vectorEncoding" because "fieldEntry" is null [lucene]

2024-08-08 Thread via GitHub
benwtrent commented on issue #13626: URL: https://github.com/apache/lucene/issues/13626#issuecomment-2275622759 @david-sitsky I haven't been able to dig further into this yet. Could you write a test that replicates the failure? I have yet to be able to replicate the failure. I *THIN

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2275507708 > To help with @rmuir's fast integer vectors issues: Once this is merged, I plan to split the VectorUtilSupport into ByteVectorUtilSupport and FloatVectorUtilSupport and use 2 getters

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on PR #13636: URL: https://github.com/apache/lucene/pull/13636#issuecomment-2275500767 As discussed before: I'd move all static constants to a package-private class, so it is initialized a bit delayed and not in class loading of VectorizationProvider. To help wit

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
uschindler commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1709095157 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -16,20 +16,56 @@ */ package org.apache.lucene.internal

Re: [PR] Compute facets while collecting [lucene]

2024-08-08 Thread via GitHub
epotyom commented on code in PR #13568: URL: https://github.com/apache/lucene/pull/13568#discussion_r1709023544 ## lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/cutters/ranges/LongRangeFacetCutter.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708960612 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/PanamaVectorizationProvider.java: ## @@ -79,4 +102,18 @@ public VectorUtilSupport getVectorUtilSuppo

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708959969 ## lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultPostingDecodingUtil.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708943154 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Optimize decoding blocks of postings using the vector API. [lucene]

2024-08-08 Thread via GitHub
jpountz commented on code in PR #13636: URL: https://github.com/apache/lucene/pull/13636#discussion_r1708936620 ## lucene/core/src/java21/org/apache/lucene/internal/vectorization/MemorySegmentPostingDecodingUtil.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software F