Re: tlog replay

2015-10-08 Thread Rallavagu
Erick, Actually, configured autocommit to 15 seconds and openSearcher is set to false. Neither 2 nor 3 happened. However, softCommit is set to 10 min. ${solr.autoCommit.maxTime:15000} false Working on upgrading to 5.3 which will take a bit of time and trying to get this

Re: tlog replay

2015-10-08 Thread Erick Erickson
right, so the scenario is 1> somehow you didn't do a hard commit (openSearcher=true or false doesn't matter) for a really long time while indexing. 2> Solr abnormally terminated. 3> When Solr started back up it replayed the entire log. How <1> happened is the mystery though. With a hard commit (au

Re: tlog replay

2015-10-08 Thread Rallavagu
As a follow up. Eventually the tlog file is disappeared (could not track the time it took to clear out completely). However, following messages were noticed in follower's log. 5120638 [recoveryExecutor-14-thread-2] WARN org.apache.solr.update.UpdateLog – Starting log replay tlog On 10/7/1

Re: tlog replay

2015-10-07 Thread Erick Erickson
The only way I can account for such a large file off the top of my head is if, for some reason, the Solr on the node somehow was failing to index documents and kept adding them to the log for a lnnn time. But how that would happen without the node being in recovery mode I'm not sure. I mean

Re: tlog replay

2015-10-07 Thread Rallavagu
Thanks Erick. Eventually, followers caught up but the 14G tlog file still persists and they are healthy. Is there anything to look for? Will monitor and see how long will it take before it disappears. Evaluating move to Solr 5.3. On 10/7/15 7:51 PM, Erick Erickson wrote: Uhm, that's very we

Re: tlog replay

2015-10-07 Thread Erick Erickson
Uhm, that's very weird. Updates are not applied from the tlog. Rather the raw doc is forwarded to the replica which both indexes the doc and writes it to the local tlog. So having a 14G tlog on a follower but a small tlog on the leader is definitely strange, especially if it persists over time. I

Re: Tlog replay

2015-07-08 Thread Yonik Seeley
On Wed, Jul 8, 2015 at 12:31 PM, Summer Shire wrote: > Thanks Alessandro ! > > Any idea on why I couldn't curl the solr core and pass the flag param ? These flags are for internal use only. Solr sets them, the client doesn't. -Yonik

Re: Tlog replay

2015-07-08 Thread Summer Shire
Thanks Alessandro ! Any idea on why I couldn't curl the solr core and pass the flag param ? > On Jul 8, 2015, at 7:12 AM, Alessandro Benedetti > wrote: > > Hi Summer, > > If you take a look to the CommitUpdateCommand class, you will notice no > Flag is in there. > > // this is the toString

Re: Tlog replay

2015-07-08 Thread Alessandro Benedetti
Hi Summer, If you take a look to the CommitUpdateCommand class, you will notice no Flag is in there. // this is the toString for example @Override public String toString() { return super.toString() + ",optimize="+optimize +",openSearcher="+openSearcher +",waitSearcher="+wai