Chris, Try to index few dummy documents and analyse if the tlogs are getting cleared or not. Ideally on the restart, it clears everything and keeps max 2 tlog per data folder.
Amrit Sarkar Search Engineer Lucidworks, Inc. 415-589-9269 www.lucidworks.com Twitter http://twitter.com/lucidworks LinkedIn: https://www.linkedin.com/in/sarkaramrit2 Medium: https://medium.com/@sarkaramrit2 On Tue, Apr 17, 2018 at 11:52 PM, Chris Troullis <cptroul...@gmail.com> wrote: > Hi Amrit, thanks for the reply. > > I shut down all of the nodes on the source cluster after the buffer was > disabled, and there was no change to the tlogs. > > On Tue, Apr 17, 2018 at 12:20 PM, Amrit Sarkar <sarkaramr...@gmail.com> > wrote: > > > Chris, > > > > After disabling the buffer on source, kind shut down all the nodes of > > source cluster first and then start them again. The tlogs will be removed > > accordingly. BTW CDCR doesn't abide by 100 numRecordsToKeep or 10 > numTlogs. > > > > Amrit Sarkar > > Search Engineer > > Lucidworks, Inc. > > 415-589-9269 > > www.lucidworks.com > > Twitter http://twitter.com/lucidworks > > LinkedIn: https://www.linkedin.com/in/sarkaramrit2 > > Medium: https://medium.com/@sarkaramrit2 > > > > On Tue, Apr 17, 2018 at 8:58 PM, Susheel Kumar <susheel2...@gmail.com> > > wrote: > > > > > DISABLEBUFFER on source cluster would solve this problem. > > > > > > On Tue, Apr 17, 2018 at 9:29 AM, Chris Troullis <cptroul...@gmail.com> > > > wrote: > > > > > > > Hi, > > > > > > > > We are attempting to use CDCR with solr 7.2.1 and are experiencing > odd > > > > behavior with transaction logs. My understanding is that by default, > > solr > > > > will keep a maximum of 10 tlog files or 100 records in the tlogs. I > > > assume > > > > that with CDCR, the records will not be removed from the tlogs until > it > > > has > > > > been confirmed that they have been replicated to the other cluster. > > > > However, even when replication has finished and the CDCR queue sizes > > are > > > 0, > > > > we are still seeing large numbers (50+) and large sizes (over a GB) > of > > > > tlogs sitting on the nodes. > > > > > > > > We are hard committing once per minute. > > > > > > > > Doing a lot of reading on the mailing list, I see that a lot of > people > > > were > > > > pointing to buffering being enabled as the cause for some of these > > > > transaction log issues. However, we have disabled buffering on both > the > > > > source and target clusters, and are still seeing the issues. > > > > > > > > Also, while some of our indexes replicate very rapidly (millions of > > > > documents in minutes), other smaller indexes are crawling. If we > > restart > > > > CDCR on the nodes then it finishes almost instantly. > > > > > > > > Any thoughts on these behaviors? > > > > > > > > Thanks, > > > > > > > > Chris > > > > > > > > > >
