Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-18 Thread via GitHub
jpountz commented on PR #13449: URL: https://github.com/apache/lucene/pull/13449#issuecomment-2175358948 FWIW I'm trying to use #11432 as a meta issue for sparse indexing and started listing tasks that I think we (ideally) need to complete to be in a good state for 9.0. -- This is an aut

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-13 Thread via GitHub
iverase merged PR #13449: URL: https://github.com/apache/lucene/pull/13449 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-12 Thread via GitHub
iverase commented on PR #13449: URL: https://github.com/apache/lucene/pull/13449#issuecomment-2162425480 If there are no more comments, I will be merging this PR tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-07 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1631012311 ## lucene/core/src/java/org/apache/lucene/index/FieldInfo.java: ## @@ -289,6 +302,24 @@ static void verifySameDocValuesType( } } + /** + * Verify that the

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-06 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1629301507 ## lucene/core/src/java/org/apache/lucene/index/FieldInfo.java: ## @@ -289,6 +302,24 @@ static void verifySameDocValuesType( } } + /** + * Verify that the

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-06 Thread via GitHub
easyice commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1629013375 ## lucene/core/src/java/org/apache/lucene/index/FieldInfo.java: ## @@ -289,6 +302,24 @@ static void verifySameDocValuesType( } } + /** + * Verify that the

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-06 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1629012824 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -197,6 +215,17 @@ private NumericEntry readNumeric(IndexInput meta) thr

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-06 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1628996406 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1749,4 +1781,88 @@ long getLongValue(long index) throws IOException {

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
benwtrent commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1628362719 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1749,4 +1781,88 @@ long getLongValue(long index) throws IOException

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627603577 ## lucene/core/src/java/org/apache/lucene/index/DocValuesSkipper.java: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627604017 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1690,4 +1722,78 @@ long getLongValue(long index) throws IOException {

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
gf2121 commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627521914 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1690,4 +1722,78 @@ long getLongValue(long index) throws IOException {

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
ChrisHegarty commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627440236 ## lucene/core/src/java/org/apache/lucene/index/DocValuesSkipper.java: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
ChrisHegarty commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627434696 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1690,4 +1722,78 @@ long getLongValue(long index) throws IOExcepti

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
jpountz commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627225989 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1690,4 +1722,78 @@ long getLongValue(long index) throws IOException {

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627219039 ## lucene/core/src/java/org/apache/lucene/index/DocValuesSkipper.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
iverase commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1627219921 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1690,4 +1722,78 @@ long getLongValue(long index) throws IOException {

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-05 Thread via GitHub
ChrisHegarty commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1626571087 ## lucene/core/src/java/org/apache/lucene/index/DocValuesSkipper.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or m

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-04 Thread via GitHub
ChrisHegarty commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1626571087 ## lucene/core/src/java/org/apache/lucene/index/DocValuesSkipper.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or m

Re: [PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-04 Thread via GitHub
ChrisHegarty commented on code in PR #13449: URL: https://github.com/apache/lucene/pull/13449#discussion_r1626526797 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java: ## @@ -1690,4 +1722,78 @@ long getLongValue(long index) throws IOExcepti

[PR] Sparse index: optional skip list on top of doc values [lucene]

2024-06-04 Thread via GitHub
iverase opened a new pull request, #13449: URL: https://github.com/apache/lucene/pull/13449 Speaking to Adrien about how a sparse index would look like in lucene, he suggested that the sparse indexing does not need to be a new format bit an additional responsibility if `DocValuesFormat`.