RE: CDCR - how to deal with the transaction log files

2017-07-28 Thread Xie, Sean
ul 28, 2017, 4:01 PM To: solr-user@lucene.apache.org mailto:solr-user@lucene.apache.org>> Cc: jmy...@wayfair.com mailto:jmy...@wayfair.com>> Subject: [EXTERNAL] RE: CDCR - how to deal with the transaction log files Amrit, Problem solved! My biggest mistake was in my SOURCE-side con

RE: CDCR - how to deal with the transaction log files

2017-07-28 Thread Patrick Hoeffel
rit Sarkar [mailto:sarkaramr...@gmail.com] Sent: Friday, July 21, 2017 7:21 AM To: solr-user@lucene.apache.org Cc: jmy...@wayfair.com Subject: Re: CDCR - how to deal with the transaction log files Patrick, Yes! You created default UpdateLog which got written to a disk and then you changed it

Re: CDCR - how to deal with the transaction log files

2017-07-21 Thread Amrit Sarkar
> -Original Message- > From: jmyatt [mailto:jmy...@wayfair.com] > Sent: Wednesday, July 12, 2017 4:49 PM > To: solr-user@lucene.apache.org > Subject: Re: CDCR - how to deal with the transaction log files > > glad to hear you found your solution! I have been combing ov

RE: CDCR - how to deal with the transaction log files

2017-07-20 Thread Patrick Hoeffel
ffel
 Senior Software Engineer (Direct)  719-452-7371 (Mobile) 719-210-3706 patrick.hoef...@polarisalpha.com PolarisAlpha.com  -Original Message- From: jmyatt [mailto:jmy...@wayfair.com] Sent: Wednesday, July 12, 2017 4:49 PM To: solr-user@lucene.apache.org Subject: Re: CDCR - how to deal

Re: CDCR - how to deal with the transaction log files

2017-07-17 Thread Susheel Kumar
I just voted for https://issues.apache.org/jira/browse/SOLR-11069 to get it resolved, as we are discussing to start using CDCR soon. On Fri, Jul 14, 2017 at 5:21 PM, Varun Thacker wrote: > https://issues.apache.org/jira/browse/SOLR-11069 is tracking why is > LASTPROCESSEDVERSION=-1 > on the sour

Re: CDCR - how to deal with the transaction log files

2017-07-14 Thread Varun Thacker
https://issues.apache.org/jira/browse/SOLR-11069 is tracking why is LASTPROCESSEDVERSION=-1 on the source cluster always On Fri, Jul 14, 2017 at 11:46 AM, jmyatt wrote: > Thanks for the suggestion - tried that today and still no luck. Time to > write a script to naively / blindly delete old log

Re: CDCR - how to deal with the transaction log files

2017-07-14 Thread jmyatt
Thanks for the suggestion - tried that today and still no luck. Time to write a script to naively / blindly delete old logs and run that in cron. *sigh* -- View this message in context: http://lucene.472066.n3.nabble.com/CDCR-how-to-deal-with-the-transaction-log-files-tp4345062p4346138.html Se

Re: CDCR - how to deal with the transaction log files

2017-07-12 Thread Xie, Sean
and tlogs again. -- Thank you Sean From: jmyatt mailto:jmy...@wayfair.com>> Date: Wednesday, Jul 12, 2017, 6:58 PM To: solr-user@lucene.apache.org mailto:solr-user@lucene.apache.org>> Subject: [EXTERNAL] Re: CDCR - how to deal with the transaction log files glad to hear you found y

Re: CDCR - how to deal with the transaction log files

2017-07-12 Thread jmyatt
glad to hear you found your solution! I have been combing over this post and others on this discussion board many times and have tried so many tweaks to configuration, order of steps, etc, all with absolutely no success in getting the Source cluster tlogs to delete. So incredibly frustrating. If

Re: CDCR - how to deal with the transaction log files

2017-07-10 Thread Xie, Sean
; We have been experiencing this same issue for months now, with > version > > 6.2. No solution to date. > > > > -----Original Message----- > > From: Xie, Sean [mailto:sean@finra.org] > > Sent: Sunday, July 0

Re: CDCR - how to deal with the transaction log files

2017-07-10 Thread Varun Thacker
-----Original Message- > > From: Xie, Sean [mailto:sean@finra.org] > > Sent: Sunday, July 09, 2017 9:41 PM > > To: solr-user@lucene.apache.org > > Subject: [EXTERNAL] Re: CDCR - how to deal with the transaction &g

Re: CDCR - how to deal with the transaction log files

2017-07-10 Thread Xie, Sean
09, 2017 9:41 PM > To: solr-user@lucene.apache.org > Subject: [EXTERNAL] Re: CDCR - how to deal with the transaction log > files > > Did another round of testing, the tlog on target cluster is cleaned up > once the hard commit is t

Re: CDCR - how to deal with the transaction log files

2017-07-10 Thread Varun Thacker
> From: Xie, Sean [mailto:sean@finra.org] > Sent: Sunday, July 09, 2017 9:41 PM > To: solr-user@lucene.apache.org > Subject: [EXTERNAL] Re: CDCR - how to deal with the transaction log > files > > Did another round of testing, the tlog on target cluster is cl

RE: CDCR - how to deal with the transaction log files

2017-07-10 Thread Xie, Sean
th version 6.2. No solution to date. -Original Message- From: Xie, Sean [mailto:sean@finra.org] Sent: Sunday, July 09, 2017 9:41 PM To: solr-user@lucene.apache.org Subject: [EXTERNAL] Re: CDCR - how to deal with the transaction log files Did anothe

RE: CDCR - how to deal with the transaction log files

2017-07-10 Thread Xie, Sean
apache.org Subject: [EXTERNAL] Re: CDCR - how to deal with the transaction log files Did another round of testing, the tlog on target cluster is cleaned up once the hard commit is triggered. However, on source cluster, the tlog files stay there and never gets cleaned up. N

RE: CDCR - how to deal with the transaction log files

2017-07-10 Thread Michael McCarthy
We have been experiencing this same issue for months now, with version 6.2. No solution to date. -Original Message- From: Xie, Sean [mailto:sean@finra.org] Sent: Sunday, July 09, 2017 9:41 PM To: solr-user@lucene.apache.org Subject: [EXTERNAL] Re: CDCR - how to deal with the

Re: CDCR - how to deal with the transaction log files

2017-07-09 Thread Xie, Sean
Did another round of testing, the tlog on target cluster is cleaned up once the hard commit is triggered. However, on source cluster, the tlog files stay there and never gets cleaned up. Not sure if there is any command to run manually to trigger the updateLogSynchronizer. The updateLogSynchron

Re: CDCR - how to deal with the transaction log files

2017-07-08 Thread Xie, Sean
I have monitored the CDCR process for a while, the updates are actively sent to the target without a problem. However the tlog size and files count are growing everyday, even when there is 0 updates to sent, the tlog stays there: Following is from the action=queues command, and you can see after

Re: CDCR - how to deal with the transaction log files

2017-07-08 Thread Erick Erickson
This should not be the case if you are actively sending updates to the target cluster. The tlog is used to store unsent updates, so if the connection is broken for some time, the target cluster will have a chance to catch up. If you don't have the remote DC online and do not intend to bring it onl