Not quite, 600000. The polling interval is half the commit interval.... This has always bothered me a little bit, I wonder at the utility of a config param. We already have old-style replication with a configurable polling interval. Under very heavy indexing loads, it seems to me that either the tlogs will grow quite large or we'll be pulling a lot of unnecessary segments across the wire, segments that'll soon be merged away and the merged segment re-pulled.
Apparently, though, nobody's seen this "in the wild", so it's theoretical at this point. On Sun, Dec 9, 2018 at 1:48 AM Vadim Ivanov <vadim.iva...@spb.ntk-intourist.ru> wrote: > > Thanks, Edward, for clues. > What bothers me is newSearcher start, warming, cache clear... all that CPU > consuming stuff in my heavy-indexing scenario. > With NRT I had autoSoftCommit: <maxTime>300000</maxTime>. > So I had new Searcher no more than every 5 min on every replica. > To have more or less the same effect with TLOG - PULL collection, > I suppose, I have to have <autoCommit> : <maxTime>300000</maxTime> > (yes, I understand that newSearchers start asynchronously on leader and > replicas) > Am I right? > -- > Vadim > > > > -----Original Message----- > > From: Edward Ribeiro [mailto:edward.ribe...@gmail.com] > > Sent: Sunday, December 09, 2018 12:42 AM > > To: solr-user@lucene.apache.org > > Subject: Re: Soft commit and new replica types > > > > Some insights in the new replica types below: > > > > On Sat, December 8, 2018 08:42, Vadim Ivanov < > > vadim.iva...@spb.ntk-intourist.ru wrote: > > > > > > > > From Ref guide we have: > > > " NRT is the only type of replica that supports soft-commits..." > > > "If TLOG replica does become a leader, it will behave the same as if it > > > was a NRT type of replica." > > > Does it mean, that if we do not have NRT replicas in the cluster then > > > autoSoftCommit section in solconfig.xml Ignored completely (even on TLOG > > > leader)? > > > > > > > No, not completely. Both TLOG and PULL nodes will periodically poll the > > leader for changes in index segments' files and download those segments > > from the leader. If hard commit max time is defined in solrconfig.xml the > > polling interval of each replica will be half that value. Or else if the > > soft commit max time is defined then the replicas will use half the soft > > commit max time as the interval. If neither are defined then the poll > > interval will be 3 seconds (hard coded). See here: > > https://github.com/apache/lucene- > > solr/blob/75b183196798232aa6f2dcaaaab117f309119053/solr/core/src/java/o > > rg/apache/solr/cloud/ReplicateFromLeader.java#L68-L77 > > > > If the TLOG is the leader it will index locally and append the doc to > > transaction log as a NRT node would do as well as it will synchronously > > replicate the data to other TLOG replicas' transaction logs (PULL nodes > > don't have transaction logs). But TLOG/PULL replicas doesn't support soft > > commits nor real time gets, afaik. > > > > > > > > > > <autoSoftCommit> > > > <maxTime>60000</maxTime> > > > </autoSoftCommit> > > > > > > Should we say that in autoCommit section openSearcher is always true in > > > that case? > > > > > > <autoCommit> > > <maxDocs>10000</maxDocs> > > <maxTime>30000</maxTime> > > <maxSize>512m</maxSize> > > <openSearcher>false</openSearcher> > > </autoCommit> > > > > Does it mean that new Searcher always starts on all replicas when hard > > commit happens on leader? > > > > > > Nope. Or at least, the searcher is not synchronously created. Each non > > leader replica will periodically fetch the index changes from the leader > > and open a new searcher to reflect those changes as seen here: > > https://github.com/apache/lucene- > > solr/blob/75b183196798232aa6f2dcaaaab117f309119053/solr/core/src/java/o > > rg/apache/solr/handler/IndexFetcher.java#L653 > > But it's important to note that the potential delay between the leader's > > hard commit and the other replicas fetching those changes from the leader > > and opening a new searcher to reflect latest changes. > > > > PS: I am still digging these new replica types so I can have misunderstood > > or missed some aspect of it. > > > > Regards, > > Edward >
