Thanks Eric. My index is near real time and frequently updated. I checked this page https://lucene.apache.org/solr/guide/8_1/uploading-data-with-index-handlers.html#xml-update-commands and using forceMerge/expungeDeletes are NOT recommended.
So I was hoping that the change in mergePolicyFactory will affect the segments with high percent of deletes as part of the REGULAR segment merging cycles. Is my understanding correct? On Fri, Oct 23, 2020 at 9:47 AM Erick Erickson <erickerick...@gmail.com> wrote: > Just go ahead and optimize/forceMerge, but do _not_ optimize to one > segment. Or you can expungeDeletes, that will rewrite all segments with > more than 10% deleted docs. As of Solr 7.5, these operations respect the 5G > limit. > > See: https://lucidworks.com/post/solr-and-optimizing-your-index-take-ii/ > > Best > Erick > > On Fri, Oct 23, 2020, 12:36 Moulay Hicham <maratusa.t...@gmail.com> wrote: > > > Hi, > > > > I am using solr 8.1 in production. We have about 30%-50% of deleted > > documents in some old segments that were merged a year ago. > > > > These segments size is about 5GB. > > > > I was wondering why these segments have a high % of deleted docs and > found > > out that they are NOT being candidates for merging because the > > default TieredMergePolicy maxMergedSegmentMB is 5G. > > > > So I have modified the TieredMergePolicyFactory config as below to > > lower the delete docs % > > > > <mergePolicyFactory > class="org.apache.solr.index.TieredMergePolicyFactory"> > > <int name="maxMergeAtOnce">10</int> > > <int name="segmentsPerTier">10</int> > > <double name="maxMergedSegmentMB">12000</double> > > <double name="deletesPctAllowed">20</double> > > </mergePolicyFactory> > > > > > > Do you see any issues with increasing the max merged segment to 12GB and > > lowered the deletedPctAllowed to 20%? > > > > Thanks, > > > > Moulay > > >
