[
https://issues.apache.org/jira/browse/LUCENE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17349619#comment-17349619
]
Adrien Grand commented on LUCENE-9539:
--------------------------------------
+1 on splitting. I had another issue when trying to use this class, which was
that it was super slow (as in many times slower than fully reindexing) due to
stored fields. So splitting could help there as well as we could rewrite stored
fields without any compression like we do at flush time, and the splitting
would help keep the amount of temporary disk space we use for uncompressed
stored fields under control.
> Improve memory footprint of SortingCodecReader
> ----------------------------------------------
>
> Key: LUCENE-9539
> URL: https://issues.apache.org/jira/browse/LUCENE-9539
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Simon Willnauer
> Priority: Major
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> SortingCodecReader is a very memory heavy since it needs to re-sort and load
> large parts of the index into memory. We can try to make it more efficient by
> using more compact internal data-structures, remove the caches it uses
> provided we define it's usage as a merge only reader wrapper. Ultimately we
> need to find a way to allow the reader or some other structure to minimize
> its heap memory. One way is to slice existing readers and merge them in
> multiple steps. There will be multiple steps towards a more useable version
> of this class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
