bq. , after getting new segments from the leader the follower replica will still apply the hard/soft commit? As was described in one of the videos below, follower tlog replica look for max docid in received new segments and purge its transaction log of older records. Than it starts new searcher(it may be called soft commit). -- Vadim
> -----Original Message----- > From: Edward Ribeiro [mailto:edward.ribe...@gmail.com] > Sent: Thursday, December 13, 2018 8:27 PM > To: solr-user@lucene.apache.org > Subject: Re: Soft commit and new replica types > > Hi Tomás, > > No, I am not seeing reloads. I am trying to understand the interactions > between hard commit, soft commit, transaction log update with a TLOG > cluster for both leader and follower replicas. For example, after getting > new segments from the leader the follower replica will still apply the > hard/soft commit? > > PS: congratulations on the Berlin Buzzwords' talk. :) > > Thanks! > > On Mon, Dec 10, 2018 at 9:24 PM Tomás Fernández Löbbe > <tomasflo...@gmail.com> > wrote: > > > I think this is a good point. The tricky part is that if TLOG replicas > > don't replicate often, their transaction logs will get too big too, so you > > want the replication interval of TLOG replicas to be tied to the > > auto(hard)Commit interval (by default at least). If you are using them for > > search, you may also not want to open a searcher for each fetch... for PULL > > replicas, maybe the best way is to use the autoSoftCommit interval to > > define the polling interval. That said, I'm not sure using different > > configurations is a good idea, some people may be mixing TLOG and PULL > and > > querying them both alike. > > > > In the meantime, if you have different hosts for TLOG and PULL replicas, > > one workaround you can have is to define the autoCommit time with a > system > > property, and use different properties for TLOGs vs PULL nodes. > > > > > There is no commit on TLOG/PULL follower replicas, only on the leader. > > > Followers fetch the segments and **reload the core** every 150 seconds > > > > Edward, "reload" shouldn't really happen in regular TLOG/PULL fetches. Are > > you seeing reloads? > > > > On Mon, Dec 10, 2018 at 4:41 PM Erick Erickson <erickerick...@gmail.com> > > wrote: > > > > > bq. but not every poll attempt they fetch new segment from the leader > > > > > > Ah, right. Ignore my comment. Commit will only occur on the followers > > > when there are new segments to pull down, so your'e right, roughly > > > every second poll would commit find things to bring down and open a > > > new searcher......... > > > On Sun, Dec 9, 2018 at 4:14 PM Edward Ribeiro > <edward.ribe...@gmail.com> > > > wrote: > > > > > > > > Hi Vadim, > > > > > > > > There is no commit on TLOG/PULL follower replicas, only on the leader. > > > > Followers fetch the segments and **reload the core** every 150 seconds > > > (if > > > > there were new segments, I suppose). Yeah, followers don't pay the CPU > > > > price of indexing, but there are still cache invalidation, autowarming, > > > > etc, in addition to network and IO demand. Is that ritht, Erick? > > > > > > > > Besides that, Erick is pointing out that under a heavy indexing > > workload > > > > you could either have: > > > > > > > > 1. Very large transaction logs; > > > > > > > > 2. Very large numbers of segments. If that is the case, you could have > > > the > > > > following scenario numerous times: > > > > 2.1. follower replica downloads segment A and B from leader; > > > > 2.2 leader merges segments A + B into C; > > > > 2.3. follower replicas discard A and B and download C on next poll; > > > > > > > > Under the second condition followers needlessly downloaded segments > > that > > > > would eventually be merged. > > > > > > > > IMO, you should carefully evaluate if the use of TLOG/PULL is really > > > > recommended for your cluster setup, plus indexing and querying > > workload. > > > > You can very much stay with a NRT setup if it suits you better. The > > > videos > > > > below provide a nice set of hints for when to choose between NRT or > > some > > > > combination of TLOG and PULL. > > > > > > > > https://youtu.be/XIb8X3MwVKc > > > > > > > > https://youtu.be/dkWy2ykzAv0 > > > > > > > > https://youtu.be/XqfTjd9KDWU > > > > > > > > Regards, > > > > Edward > > > > > > > > Em dom, 9 de dez de 2018 16:56, <vadim.iva...@spb.ntk-intourist.ru > > > escreveu: > > > > > > > > > > > > > > If hard commit max time is 300 sec then commit happens every 300 > sec > > > on > > > > > tlog leader. And new segments pop up on the leader every 300 sec, > > > during > > > > > indexing. Polling interval on other replicas 150 sec, but not every > > > poll > > > > > attempt they fetch new segment from the leader, afaiu. Erick, do you > > > mean > > > > > that on all other tlog replicas(not leaders) commit occurs every > > poll? > > > > > воскресенье, 09 декабря 2018г., 19:21 +03:00 от Erick Erickson > > > > > erickerick...@gmail.com : > > > > > > > > > > >Not quite, 600000. The polling interval is half the commit > > > interval.... > > > > > > > > > > > >This has always bothered me a little bit, I wonder at the utility > > of a > > > > > >config param. We already have old-style replication with a > > > > > >configurable polling interval. Under very heavy indexing loads, it > > > > > >seems to me that either the tlogs will grow quite large or we'll be > > > > > >pulling a lot of unnecessary segments across the wire, segments > > > > > >that'll soon be merged away and the merged segment re-pulled. > > > > > > > > > > > >Apparently, though, nobody's seen this "in the wild", so it's > > > > > >theoretical at this point. > > > > > >On Sun, Dec 9, 2018 at 1:48 AM Vadim Ivanov > > > > > < vadim.iva...@spb.ntk-intourist.ru> wrote: > > > > > > > > > > > > Thanks, Edward, for clues. > > > > > > What bothers me is newSearcher start, warming, cache clear... all > > > that > > > > > CPU consuming stuff in my heavy-indexing scenario. > > > > > > With NRT I had autoSoftCommit: 300000 . > > > > > > So I had new Searcher no more than every 5 min on every replica. > > > > > > To have more or less the same effect with TLOG - PULL collection, > > > > > > I suppose, I have to have : 300000 > > > > > > (yes, I understand that newSearchers start asynchronously on leader > > > and > > > > > replicas) > > > > > > Am I right? > > > > > > -- > > > > > > Vadim > > > > > > > > > > > > > > > > > >> -----Original Message----- > > > > > >> From: Edward Ribeiro [mailto:edward.ribe...@gmail.com] > > > > > >> Sent: Sunday, December 09, 2018 12:42 AM > > > > > >> To: solr-user@lucene.apache.org > > > > > >> Subject: Re: Soft commit and new replica types > > > > > >> > > > > > >> Some insights in the new replica types below: > > > > > >> > > > > > >> On Sat, December 8, 2018 08:42, Vadim Ivanov < > > > > > >> vadim.iva...@spb.ntk-intourist.ru wrote: > > > > > >> > > > > > >>> > > > > > >>> From Ref guide we have: > > > > > >>> " NRT is the only type of replica that supports soft-commits..." > > > > > >>> "If TLOG replica does become a leader, it will behave the same as > > > if it > > > > > >>> was a NRT type of replica." > > > > > >>> Does it mean, that if we do not have NRT replicas in the cluster > > > then > > > > > >>> autoSoftCommit section in solconfig.xml Ignored completely (even > > on > > > > > TLOG > > > > > >>> leader)? > > > > > >>> > > > > > >> > > > > > >> No, not completely. Both TLOG and PULL nodes will periodically > > poll > > > the > > > > > >> leader for changes in index segments' files and download those > > > segments > > > > > >> from the leader. If hard commit max time is defined in > > > solrconfig.xml > > > > > the > > > > > >> polling interval of each replica will be half that value. Or else > > > if the > > > > > >> soft commit max time is defined then the replicas will use half > > the > > > soft > > > > > >> commit max time as the interval. If neither are defined then the > > > poll > > > > > >> interval will be 3 seconds (hard coded). See here: > > > > > >> https://github.com/apache/lucene- > > > > > >> > > > > solr/blob/75b183196798232aa6f2dcaaaab117f309119053/solr/core/src/java/o > > > > > >> rg/apache/solr/cloud/ReplicateFromLeader.java#L68-L77 > > > > > >> > > > > > >> If the TLOG is the leader it will index locally and append the doc > > > to > > > > > >> transaction log as a NRT node would do as well as it will > > > synchronously > > > > > >> replicate the data to other TLOG replicas' transaction logs (PULL > > > nodes > > > > > >> don't have transaction logs). But TLOG/PULL replicas doesn't > > support > > > > > soft > > > > > >> commits nor real time gets, afaik. > > > > > >> > > > > > >>> > > > > > >> > > > > > >>> > > > > > >>> 60000 > > > > > >>> > > > > > >>> > > > > > >>> Should we say that in autoCommit section openSearcher is always > > > true in > > > > > >>> that case? > > > > > >> > > > > > >> > > > > > >> > > > > > >> 10000 > > > > > >> 30000 > > > > > >> 512m > > > > > >> false > > > > > >> > > > > > >> > > > > > >> Does it mean that new Searcher always starts on all replicas when > > > hard > > > > > >> commit happens on leader? > > > > > >> > > > > > >> > > > > > >> Nope. Or at least, the searcher is not synchronously created. Each > > > non > > > > > >> leader replica will periodically fetch the index changes from the > > > leader > > > > > >> and open a new searcher to reflect those changes as seen here: > > > > > >> https://github.com/apache/lucene- > > > > > >> > > > > solr/blob/75b183196798232aa6f2dcaaaab117f309119053/solr/core/src/java/o > > > > > >> rg/apache/solr/handler/IndexFetcher.java#L653 > > > > > >> But it's important to note that the potential delay between the > > > leader's > > > > > >> hard commit and the other replicas fetching those changes from the > > > > > leader > > > > > >> and opening a new searcher to reflect latest changes. > > > > > >> > > > > > >> PS: I am still digging these new replica types so I can have > > > > > misunderstood > > > > > >> or missed some aspect of it. > > > > > >> > > > > > >> Regards, > > > > > >> Edward > > > > > > > > > > > > > > > >
