On 1/28/2019 10:14 AM, Zheng Lin Edwin Yeo wrote:
We have the following TieredMergePolicyFactory configuration in our solrconfig,xml<mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory"> <int name="maxMergeAtOnce">10</int> <int name="maxMergeAtOnceExplicit">10</int> <int name="segmentsPerTier">10</int>
These three settings are the really important ones. Except for maxMergeAtOnceExplicit, you have these at the default settings. The default for maxMergeAtOnceExplicit is 30 ... and you shouldn't lower it without a really good reason. It mostly comes into play during an optimize ... when you lower it, optimizes may take longer than normal. It won't be able to merge as many segments at the same time, so the number of passes required to complete the optimize could increase.
The most important setting here is segmentsPerTier ... this does not mean you will never have more than 10 total segments, it means that at each tier, Lucene will try to keep the number of segments below 10. With a large index, you are likely to have 3 or 4 tiers, possibly more.
On an index where I spent a lot of time, my settings were, respective to yours, 35, 105, and 35. I often had more than 100 segments in those indexes. It was behaving correctly.
What could be the reason that it is not able to merge the segments to 3, with each of the segment size to be 5 GB?
It is working as designed, just not as you expected. Thanks, Shawn
