Re: [PR] Split taxonomy arrays across chunks [lucene]

stefanvodita commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444224140 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -68,25 +90,49 @@ public TaxonomyIndexArrays(IndexReader reader

Re: [PR] Split taxonomy arrays across chunks [lucene]

stefanvodita commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444224140 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -68,25 +90,49 @@ public TaxonomyIndexArrays(IndexReader reader

Re: [PR] Split taxonomy arrays across chunks [lucene]

stefanvodita commented on PR #12995: URL: https://github.com/apache/lucene/pull/12995#issuecomment-1880461956 As far as testing, can we add some unit tests that allocate more than one chunk and exercise the new functionality? Should we also run some benchmarks to understand if there's any

Re: [PR] Split taxonomy arrays across chunks [lucene]

stefanvodita commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444211059 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -68,25 +90,49 @@ public TaxonomyIndexArrays(IndexReader reader

Re: [I] Taxonomy facets: can we change massive `int[]` for parent/child/sibling tree to paged/block `int[]` to reduce RAM pressure? [lucene]

msfroh commented on issue #12989: URL: https://github.com/apache/lucene/issues/12989#issuecomment-188097 > It's very possible I'm missing something, but right now it looks to me like we haven't improved on that front. Doing shallow copies of the old array without allocating new memory w

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444073439 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -38,27 +38,49 @@ * @lucene.experimental */ class TaxonomyIndexA

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444070455 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -153,12 +203,13 @@ private void initParents(IndexReader reader, int

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444070455 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -153,12 +203,13 @@ private void initParents(IndexReader reader, int

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444069632 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -68,25 +90,49 @@ public TaxonomyIndexArrays(IndexReader reader, Tax

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444069001 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -68,25 +90,49 @@ public TaxonomyIndexArrays(IndexReader reader, Tax

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444066462 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -68,25 +90,49 @@ public TaxonomyIndexArrays(IndexReader reader, Tax

Re: [PR] Split taxonomy arrays across chunks [lucene]

msfroh commented on code in PR #12995: URL: https://github.com/apache/lucene/pull/12995#discussion_r1444066152 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/TaxonomyIndexArrays.java: ## @@ -38,27 +38,49 @@ * @lucene.experimental */ class TaxonomyIndexA

[PR] Avoid reset BlockDocsEnum#freqBuffer when indexHasFreq is false [lucene]

easyice opened a new pull request, #12997: URL: https://github.com/apache/lucene/pull/12997 As discussion in https://github.com/apache/lucene/pull/12954, this is the second implementation can be considered. this change yielded a 13% speedup for `PKLookup`. -- This is an automated message

Re: [PR] Reduce ArrayUtil#grow in decompress [lucene]

easyice commented on code in PR #12996: URL: https://github.com/apache/lucene/pull/12996#discussion_r1443993181 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/LZ4WithPresetDictCompressionMode.java: ## @@ -128,10 +128,12 @@ public void decompress(DataInput in, int orig

Re: [PR] Reduce ArrayUtil#grow in decompress [lucene]

easyice commented on code in PR #12996: URL: https://github.com/apache/lucene/pull/12996#discussion_r1443987082 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/LZ4WithPresetDictCompressionMode.java: ## @@ -128,10 +128,12 @@ public void decompress(DataInput in, int orig

Re: [PR] Reduce ArrayUtil#grow in decompress [lucene]

dungba88 commented on code in PR #12996: URL: https://github.com/apache/lucene/pull/12996#discussion_r1443985033 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/LZ4WithPresetDictCompressionMode.java: ## @@ -128,10 +128,12 @@ public void decompress(DataInput in, int ori