[jira] [Commented] (LUCENE-10205) Should Packed64 use a byte[] plus VarHandles?

2022-01-20 Thread Costin Leau (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479157#comment-17479157 ] Costin Leau commented on LUCENE-10205: -- Linking an investigation that used VarHand

[jira] [Created] (LUCENE-10383) Explore moving HNSW's NeighborQueue to a radix heap

2022-01-20 Thread Adrien Grand (Jira)
Adrien Grand created LUCENE-10383: - Summary: Explore moving HNSW's NeighborQueue to a radix heap Key: LUCENE-10383 URL: https://issues.apache.org/jira/browse/LUCENE-10383 Project: Lucene - Core

[jira] [Created] (LUCENE-10384) Simplify LongHeap

2022-01-20 Thread Adrien Grand (Jira)
Adrien Grand created LUCENE-10384: - Summary: Simplify LongHeap Key: LUCENE-10384 URL: https://issues.apache.org/jira/browse/LUCENE-10384 Project: Lucene - Core Issue Type: Task Re

[GitHub] [lucene] jpountz opened a new pull request #615: LUCENE-10384: Simplify LongHeap.

2022-01-20 Thread GitBox
jpountz opened a new pull request #615: URL: https://github.com/apache/lucene/pull/615 The min/max ordering logic moves to NeighborQueue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (LUCENE-10382) Allow KnnVectorQuery to operate over a subset of liveDocs

2022-01-20 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479227#comment-17479227 ] Adrien Grand commented on LUCENE-10382: --- We have queries like ParentChildrenBlock

[jira] [Created] (LUCENE-10385) Implement Weight#count on IndexSortSortedNumericDocValuesRangeQuery.

2022-01-20 Thread Adrien Grand (Jira)
Adrien Grand created LUCENE-10385: - Summary: Implement Weight#count on IndexSortSortedNumericDocValuesRangeQuery. Key: LUCENE-10385 URL: https://issues.apache.org/jira/browse/LUCENE-10385 Project: Luc

[jira] [Commented] (LUCENE-8930) Integrate Luke app health check to nightly tests (CI)

2022-01-20 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479329#comment-17479329 ] ASF subversion and git services commented on LUCENE-8930: - Commi

[jira] [Commented] (LUCENE-7282) search APIs should take advantage of index sort by default

2022-01-20 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479331#comment-17479331 ] Adrien Grand commented on LUCENE-7282: -- One of my motivations for LUCENE-10162 was

[GitHub] [lucene] jpountz commented on a change in pull request #612: LUCENE-10002: Replace simple usages of TotalHitCountCollector with IndexSearcher#count

2022-01-20 Thread GitBox
jpountz commented on a change in pull request #612: URL: https://github.com/apache/lucene/pull/612#discussion_r788730995 ## File path: lucene/core/src/test/org/apache/lucene/search/TestLRUQueryCache.java ## @@ -1174,7 +1172,7 @@ public void testDetectMutatedQueries() throws IOE

[jira] [Resolved] (LUCENE-8930) Integrate Luke app health check to nightly tests (CI)

2022-01-20 Thread Tomoko Uchida (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomoko Uchida resolved LUCENE-8930. --- Fix Version/s: 9.1 10.0 (main) Resolution: Fixed > Integrate Luke

[jira] [Commented] (LUCENE-8930) Integrate Luke app health check to nightly tests (CI)

2022-01-20 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479460#comment-17479460 ] ASF subversion and git services commented on LUCENE-8930: - Commi

[jira] [Commented] (LUCENE-8930) Integrate Luke app health check to nightly tests (CI)

2022-01-20 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479466#comment-17479466 ] ASF subversion and git services commented on LUCENE-8930: - Commi

[jira] [Commented] (LUCENE-10382) Allow KnnVectorQuery to operate over a subset of liveDocs

2022-01-20 Thread Michael Sokolov (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479487#comment-17479487 ] Michael Sokolov commented on LUCENE-10382: -- If we go with a {{{}Query{}}}-base

[GitHub] [lucene] gsmiller commented on a change in pull request #615: LUCENE-10384: Simplify LongHeap.

2022-01-20 Thread GitBox
gsmiller commented on a change in pull request #615: URL: https://github.com/apache/lucene/pull/615#discussion_r788984286 ## File path: lucene/core/src/java/org/apache/lucene/util/hnsw/NeighborQueue.java ## @@ -29,17 +29,32 @@ */ public class NeighborQueue { + private sta

[GitHub] [lucene] jpountz commented on a change in pull request #615: LUCENE-10384: Simplify LongHeap.

2022-01-20 Thread GitBox
jpountz commented on a change in pull request #615: URL: https://github.com/apache/lucene/pull/615#discussion_r789000738 ## File path: lucene/core/src/java/org/apache/lucene/util/LongHeap.java ## @@ -17,27 +17,16 @@ package org.apache.lucene.util; /** - * A heap that stores

[GitHub] [lucene] jpountz commented on a change in pull request #615: LUCENE-10384: Simplify LongHeap.

2022-01-20 Thread GitBox
jpountz commented on a change in pull request #615: URL: https://github.com/apache/lucene/pull/615#discussion_r789003876 ## File path: lucene/core/src/java/org/apache/lucene/util/hnsw/NeighborQueue.java ## @@ -29,17 +29,32 @@ */ public class NeighborQueue { + private stat

[GitHub] [lucene] mdmarshmallow commented on a change in pull request #611: LUCENE-9952: Fix dim count inaccuracies in SSDV faceting when a dim is multi-valued

2022-01-20 Thread GitBox
mdmarshmallow commented on a change in pull request #611: URL: https://github.com/apache/lucene/pull/611#discussion_r788936337 ## File path: lucene/facet/src/java/org/apache/lucene/facet/FacetsConfig.java ## @@ -494,10 +498,16 @@ private void processSSDVFacetFields(

[GitHub] [lucene] mdmarshmallow commented on a change in pull request #611: LUCENE-9952: Fix dim count inaccuracies in SSDV faceting when a dim is multi-valued

2022-01-20 Thread GitBox
mdmarshmallow commented on a change in pull request #611: URL: https://github.com/apache/lucene/pull/611#discussion_r788936337 ## File path: lucene/facet/src/java/org/apache/lucene/facet/FacetsConfig.java ## @@ -494,10 +498,16 @@ private void processSSDVFacetFields(

[GitHub] [lucene] mdmarshmallow commented on pull request #613: LUCENE-10381: Require users to provide FacetsConfig for SSDV faceting

2022-01-20 Thread GitBox
mdmarshmallow commented on pull request #613: URL: https://github.com/apache/lucene/pull/613#issuecomment-1017805137 > This is trappy and we should require users to provide their facets config in all cases I agree with what you said here in regards to not providing a facets config.

[jira] [Commented] (LUCENE-10382) Allow KnnVectorQuery to operate over a subset of liveDocs

2022-01-20 Thread Joel Bernstein (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479613#comment-17479613 ] Joel Bernstein commented on LUCENE-10382: - I think Query makes sense as well. I

[GitHub] [lucene] gsmiller commented on a change in pull request #615: LUCENE-10384: Simplify LongHeap.

2022-01-20 Thread GitBox
gsmiller commented on a change in pull request #615: URL: https://github.com/apache/lucene/pull/615#discussion_r789085305 ## File path: lucene/core/src/java/org/apache/lucene/util/hnsw/NeighborQueue.java ## @@ -29,17 +29,32 @@ */ public class NeighborQueue { + private sta

[GitHub] [lucene] mayya-sharipova opened a new pull request #616: LUCENE-9573 Add Vectors to TestBackwardsCompatibility

2022-01-20 Thread GitBox
mayya-sharipova opened a new pull request #616: URL: https://github.com/apache/lucene/pull/616 This patch adds KNN vectors for testing backward compatible indices - Add a KnnVectorField to documents when creating a new backward compatible index - Add knn vectors search and a c

[jira] [Commented] (LUCENE-10382) Allow KnnVectorQuery to operate over a subset of liveDocs

2022-01-20 Thread Michael Sokolov (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479649#comment-17479649 ] Michael Sokolov commented on LUCENE-10382: -- > I'm a little fuzzy on the cost c

[GitHub] [lucene] msokolov commented on pull request #615: LUCENE-10384: Simplify LongHeap.

2022-01-20 Thread GitBox
msokolov commented on pull request #615: URL: https://github.com/apache/lucene/pull/615#issuecomment-1017885797 Thanks, makes sense! Do you see any performance difference? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Comment Edited] (LUCENE-10382) Allow KnnVectorQuery to operate over a subset of liveDocs

2022-01-20 Thread Michael Sokolov (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479649#comment-17479649 ] Michael Sokolov edited comment on LUCENE-10382 at 1/20/22, 8:20 PM: -

[GitHub] [lucene] gsmiller commented on a change in pull request #611: LUCENE-9952: Fix dim count inaccuracies in SSDV faceting when a dim is multi-valued

2022-01-20 Thread GitBox
gsmiller commented on a change in pull request #611: URL: https://github.com/apache/lucene/pull/611#discussion_r789158426 ## File path: lucene/facet/src/java/org/apache/lucene/facet/sortedset/ConcurrentSortedSetDocValuesFacetCounts.java ## @@ -121,12 +124,26 @@ public FacetRes

[GitHub] [lucene] gsmiller commented on pull request #611: LUCENE-9952: Fix dim count inaccuracies in SSDV faceting when a dim is multi-valued

2022-01-20 Thread GitBox
gsmiller commented on pull request #611: URL: https://github.com/apache/lucene/pull/611#issuecomment-1017937435 Thanks for having a look @mdmarshmallow. Just updated based on your feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Commented] (LUCENE-10250) Add hierarchical labels to SSDV facets

2022-01-20 Thread Marc D'Mello (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479681#comment-17479681 ] Marc D'Mello commented on LUCENE-10250: --- I added {{BrowseDateSSDVFacets}} as a ne

[GitHub] [lucene] jtibshirani opened a new pull request #617: LUCENE-10375: Write vectors to file in flush

2022-01-20 Thread GitBox
jtibshirani opened a new pull request #617: URL: https://github.com/apache/lucene/pull/617 In a previous commit, we updated HNSW merge to first write the combined segment vectors to a file, then use that file to build the graph. This commit applies the same strategy to flush, which le

[GitHub] [lucene] jtibshirani commented on pull request #617: LUCENE-10375: Write vectors to file in flush

2022-01-20 Thread GitBox
jtibshirani commented on pull request #617: URL: https://github.com/apache/lucene/pull/617#issuecomment-1017979480 Some notes: * The HNSW format no longer needs the ability to override `merge`. It's still possible though -- it seems like helpful flexibility and there's precedent with ot

[jira] [Commented] (LUCENE-10375) Speed up HNSW merge by writing combined vector data

2022-01-20 Thread Julie Tibshirani (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479731#comment-17479731 ] Julie Tibshirani commented on LUCENE-10375: --- I tried simplifying by using the

[jira] [Comment Edited] (LUCENE-10375) Speed up HNSW merge by writing combined vector data

2022-01-20 Thread Julie Tibshirani (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479731#comment-17479731 ] Julie Tibshirani edited comment on LUCENE-10375 at 1/20/22, 10:46 PM: ---

[GitHub] [lucene] msokolov commented on pull request #617: LUCENE-10375: Write vectors to file in flush

2022-01-20 Thread GitBox
msokolov commented on pull request #617: URL: https://github.com/apache/lucene/pull/617#issuecomment-1018061477 Well, it's certainly a nice cleanup. The performance results are confusing at first glance, since we now write the temp file and then copy it, but consider that now when we trave

[GitHub] [lucene] msokolov edited a comment on pull request #617: LUCENE-10375: Write vectors to file in flush

2022-01-20 Thread GitBox
msokolov edited a comment on pull request #617: URL: https://github.com/apache/lucene/pull/617#issuecomment-1018061477 Well, it's certainly a nice cleanup. The performance results are confusing at first glance, since we now write the temp file and then copy it, but consider that now when w

[jira] [Updated] (LUCENE-10336) DirectDocValueFormat: use RandomAccessSlice instead of loading giant byte[] arrays

2022-01-20 Thread spike liu (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] spike liu updated LUCENE-10336: --- Attachment: LUCENE-10336-20220121.patch > DirectDocValueFormat: use RandomAccessSlice instead of lo

[jira] [Commented] (LUCENE-10336) DirectDocValueFormat: use RandomAccessSlice instead of loading giant byte[] arrays

2022-01-20 Thread spike liu (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479773#comment-17479773 ] spike liu commented on LUCENE-10336: + first version of implementation and will cre

[GitHub] [lucene] joel-bernstein opened a new pull request #618: LUCENE-10382: Allow KnnVectorQuery to operate over a subset of liveDo…

2022-01-20 Thread GitBox
joel-bernstein opened a new pull request #618: URL: https://github.com/apache/lucene/pull/618 …cs WIP # Description Please provide a short description of the changes you're making with this pull request. # Solution Please provide a short description o

[jira] [Commented] (LUCENE-10382) Allow KnnVectorQuery to operate over a subset of liveDocs

2022-01-20 Thread Joel Bernstein (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479810#comment-17479810 ] Joel Bernstein commented on LUCENE-10382: - I put up a PR with a Query as the fi

[GitHub] [lucene] comdotwang162 commented on a change in pull request #601: LUCENE-10375: Write merged vectors to file before building graph

2022-01-20 Thread GitBox
comdotwang162 commented on a change in pull request #601: URL: https://github.com/apache/lucene/pull/601#discussion_r789345508 ## File path: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90HnswVectorsWriter.java ## @@ -110,26 +113,17 @@ @Override public vo