[
https://issues.apache.org/jira/browse/LUCENE-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540835#comment-17540835
]
Adrien Grand commented on LUCENE-10574:
---------------------------------------
The stored fields benchmark aimed at reproducing a pathological case, but I
don't think this case is uncommon.
The only thing you need to be affected by O(n^2) merges is to flush segments
that are significantly smaller than the default floor segment size of
TieredMergePolicy (2MB). We almost never see this in our benchmarks because our
indexing logic always tries to max out indexing speed, so even with the default
RAM buffer size of 16MB, the smallest segments in the index would be above 2MB.
However in the real world where there are frequent reopens, this wouldn't be
unlikely. For instance, if your documents require ~100 bytes of disk space each
in the index, and your indexing/refresh rate trigger creation of segments of
~100 documents each, then you'll end up with ~10kB flush segments and hit
pathological merges.
> Remove O(n^2) from TieredMergePolicy or change defaults to one that doesn't
> do this
> -----------------------------------------------------------------------------------
>
> Key: LUCENE-10574
> URL: https://issues.apache.org/jira/browse/LUCENE-10574
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Major
> Fix For: 9.3
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Remove {{floorSegmentBytes}} parameter, or change lucene's default to a merge
> policy that doesn't merge in an O(n^2) way.
> I have the feeling it might have to be the latter, as folks seem really wed
> to this crazy O(n^2) behavior.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
