bq. Do you recommend disabling the buffer on the source SolrCloud as well? Disable them all on both source and target IMO.
On Tue, Jun 19, 2018 at 8:50 AM, Brian Yee <b...@wayfair.com> wrote: > Thank you Erick. I am running Solr 6.6. From the documentation: > "Replicas do not need to buffer updates, and it is recommended to disable > buffer on the target SolrCloud." > > Do you recommend disabling the buffer on the source SolrCloud as well? It > looks like I already have the buffer disabled at target locations but not the > source location. Would it even make sense at the source location? > > This is what I have at the target locations: > <requestHandler name="/cdcr" class="solr.CdcrRequestHandler"> > <shardHandlerFactory class="HttpShardHandlerFactory"> > <int name="maxConnectionsPerHost">100</int> > </shardHandlerFactory> > <lst name="buffer"> > <str name="defaultState">disabled</str> > </lst> > </requestHandler> > > > -----Original Message----- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, June 19, 2018 11:00 AM > To: solr-user <solr-user@lucene.apache.org> > Subject: Re: tlogs not deleting > > Take a look at the CDCR section of your reference guide, be sure you get the > version which you can download from here: > https://archive.apache.org/dist/lucene/solr/ref-guide/ > > There's the CDCR API call you can use for in-flight disabling, and depending > on the version of Solr you can set it in solrconfig. > > Basically, buffering was there in the original CDCR to allow a larger > maintenance window, you could enable buffering and all updates were saved > until you disabled it, during which period you could do whatever you needed > with your target cluster and not lose any updates. > > Later versions can do the full sync of the index and buffering is being > removed. > > Best, > Erick > > On Tue, Jun 19, 2018 at 7:31 AM, Brian Yee <b...@wayfair.com> wrote: >> Thanks for the suggestion. Can you please elaborate a little bit about what >> DISABLEBUFFER does? The documentation is not very detailed. Is this >> something that needs to be done manually whenever this problem happens or is >> it something that we can do to fix it so it won't happen again? >> >> -----Original Message----- >> From: Susheel Kumar [mailto:susheel2...@gmail.com] >> Sent: Monday, June 18, 2018 9:12 PM >> To: solr-user@lucene.apache.org >> Subject: Re: tlogs not deleting >> >> You may have to DISABLEBUFFER in source to get rid of tlogs. >> >> On Mon, Jun 18, 2018 at 6:13 PM, Brian Yee <b...@wayfair.com> wrote: >> >>> So I've read a bunch of stuff on hard/soft commits and tlogs. As I >>> understand, after a hard commit, solr is supposed to delete old tlogs >>> depending on the numRecordsToKeep and maxNumLogsToKeep values in the >>> autocommit settings in solrconfig.xml. I am occasionally seeing solr >>> fail to do this and the tlogs just build up over time and eventually >>> we run out of disk space on the VM and this causes problems for us. >>> This does not happen all the time, only sometimes. I currently have a >>> tlog directory that has 123G worth of tlogs. The last hard commit on >>> this node was 10 minutes ago but these tlogs date back to 3 days ago. >>> >>> We have sometimes found that restarting solr on the node will get it >>> to clean up the old tlogs, but we really want to find the root cause >>> and fix it if possible so we don't keep getting disk space alerts and >>> have to adhoc restart nodes. Has anyone seen an issue like this before? >>> >>> My update handler settings look like this: >>> <updateHandler class="solr.DirectUpdateHandler2"> >>> >>> <updateLog class="solr.CdcrUpdateLog"> >>> >>> <str name="dir">${solr.ulog.dir:}</str> >>> <int name="numVersionBuckets">${solr.ulog.numVersionBuckets: >>> 65536}</int> >>> </updateLog> >>> <autoCommit> >>> <maxTime>600000</maxTime> >>> <maxDocs>250000</maxDocs> >>> <openSearcher>false</openSearcher> >>> </autoCommit> >>> <autoSoftCommit> >>> <maxTime>120000</maxTime> >>> </autoSoftCommit> >>> >>> <shardHandlerFactory class="HttpShardHandlerFactory"> >>> <int name="maxConnectionsPerHost">100</int> >>> </shardHandlerFactory> >>> >>> </updateHandler> >>>
