A side question, for heavy bulk indexing, what's the recommended setting for auto commit? As there is no query needed during the bulking indexing process, I have auto soft commit disabled. Is there any side effect if I also disable auto commit?
On Sun, Mar 10, 2019 at 10:22 PM Wei <weiwan...@gmail.com> wrote: > Thanks Erick. > > 1> TLOG replicas shouldn’t optimize on the follower. They should optimize > on the leader then replicate the entire index to the follower. > > Does that mean the follower will ignore the optimize request? Or shall I > send the optimize request only to one of the leaders? > > 2> As of Solr 7.5, optimize should not optimize to a single segment > _unless_ that segment is < 5G. See LUCENE-7976. Or you explicitly set > numSegments on the optimize command. > > -- Is the 5G limit controlled by maxMegedSegmentMB setting? In > solrconfig.xml I used these settings: > > <mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory"> > <int name="maxMergeAtOnceExplicit">100</int> > <int name="maxMergeAtOnce">10</int> > <int name="segmentsPerTier">10</int> > <double name="maxMergedSegmentMB">20480</double> > </mergePolicyFactory> > > But in the end I see multiple segments much smaller than the 20GB limit. > In 7.6 is it required to explicitly set the number of segments to 1? e.g > shall I use > > /update?optimize=true&waitSearcher=false&maxSegments=1 > > Best, > Wei > > > On Fri, Mar 8, 2019 at 12:29 PM Erick Erickson <erickerick...@gmail.com> > wrote: > >> This is very odd for at least two reasons: >> >> 1> TLOG replicas shouldn’t optimize on the follower. They should optimize >> on the leader then replicate the entire index to the follower. >> >> 2> As of Solr 7.5, optimize should not optimize to a single segment >> _unless_ that segment is < 5G. See LUCENE-7976. Or you explicitly set >> numSegments on the optimize command. >> >> So if you can reliably reproduce this, it’s probably worth a JIRA…... >> >> > On Mar 8, 2019, at 11:21 AM, Wei <weiwan...@gmail.com> wrote: >> > >> > Hi, >> > >> > RecentIy I encountered a strange issue with optimize in Solr 7.6. The >> cloud >> > is created with 4 shards with 2 Tlog replicas per shard. After batch >> index >> > update I issue an optimize command to a randomly picked replica in the >> > cloud. After a while when I check, all the non-leader Tlog replicas >> > finished optimization to a single segment, however all the leader >> replicas >> > still have multiple segments. Previously inn the all NRT replica >> cloud, I >> > see optimization is triggered on all nodes. Is the optimization process >> > different with Tlog/Pull replicas? >> > >> > Best, >> > Wei >> >>
