Re: slave is getting full synced every polling

2016-02-12 Thread Erick Erickson
If you have to stay on master/slave, then the full replication when you do this switch is probably just a price you'll have to pay. The indexes are different so to be on the safe side Solr will replicate the whole thing. Is it really that much of a problem? As Shawn says, though, much of this wou

Re: slave is getting full synced every polling

2016-02-12 Thread Shawn Heisey
On 2/12/2016 11:47 AM, Novin Novin wrote: > I think I didn't explain it quit properly. So I have situation in which > data is getting index every 20 seconds or less and I can't loose data while > indexing. I use searching a lot in website, if I have to restart my solr > machine because of kernel up

Re: slave is getting full synced every polling

2016-02-12 Thread Novin Novin
you're trying to accomplish X and asking about Y where Y is the index replication. What's X? What is the purpose of switching the master and slave and how often do you do it and why? I think I didn't explain it quit properly. So I have situation in which data is getting index every 20 seconds or l

Re: slave is getting full synced every polling

2016-02-12 Thread Erick Erickson
bq: What I have done when the problem started, I changed slave to master and master to slave. OK, other things aside, if you're really saying that every time you switch the slave and master around and restart, you get a full sync then I'd reply "don't do that". Why are you switching slave and mast

Re: slave is getting full synced every polling

2016-02-12 Thread Alessandro Benedetti
Have you customised the merge factor ? Is it aggressive ? In case a lot of merge happens, you can potentially incur in a big trasnfer of files each replication . You need to check the segments in the slave every minutes. When the replication is triggered what are the difference from the Master inde

Re: slave is getting full synced every polling

2016-02-12 Thread Novin Novin
sorry core name is wmsapp_analysis which is big core On Fri, 12 Feb 2016 at 12:01 Novin Novin wrote: > Well It started again. > > Below is are the errors from solr logging on admin ui. > Log error message in master > 2/12/2016, 11:39:24 AM null:java.lang.IllegalStateException: file: > MMapDirect

Re: slave is getting full synced every polling

2016-02-12 Thread Novin Novin
Well It started again. Below is are the errors from solr logging on admin ui. Log error message in master 2/12/2016, 11:39:24 AM null:java.lang.IllegalStateException: file: MMapDirectory@/var/solr/data/wmsapp_analysis/data/index.20160211204900750 lockFactory=org.apache.lucene.store.NativeFSLockFac

Re: slave is getting full synced every polling

2016-02-12 Thread Novin Novin
Details here are important. Do you understand what Erick was asking when he was talking about segments? The segments are the files in the index directory, which is usually data/index inside the core's instance directory. Thanks Shawn, If I am thinking right these segments also appears on core ad

Re: slave is getting full synced every polling

2016-02-12 Thread Shawn Heisey
On 2/12/2016 1:58 AM, Novin Novin wrote: > Typo? That's 60 seconds, but that's not especially interesting either way. > > Yes, I was thinking about this too and I have changed it to 59 actually. If you want the polling to occur once an hour, pollInterval will need to be set to 01:00:00 ... not 00:

Re: slave is getting full synced every polling

2016-02-12 Thread Novin Novin
Typo? That's 60 seconds, but that's not especially interesting either way. Yes, I was thinking about this too and I have changed it to 59 actually. Do the actual segment's look identical after the polling? Well no. How I am handling master slave. How we do this use sym link for master and slav

Re: slave is getting full synced every polling

2016-02-11 Thread Erick Erickson
Typo? That's 60 seconds, but that's not especially interesting either way. Do the actual segment's look identical after the polling? On Thu, Feb 11, 2016 at 1:16 PM, Novin Novin wrote: > Hi Erick, > > Below is master slave config: > > Master: > > > commit > optimize >

Re: slave is getting full synced every polling

2016-02-11 Thread Novin Novin
Hi Erick, Below is master slave config: Master: commit optimize 2 Slave: http://master:8983/solr/big_core/replication 00:00:60 username password Do you mean the Solr is restarting every minute or the polling inte

Re: slave is getting full synced every polling

2016-02-11 Thread Erick Erickson
What is your replication configuration in solrconfig.xml on both master and slave? bq: big core is doing full sync every time wherever it start (every minute). Do you mean the Solr is restarting every minute or the polling interval is 60 seconds? The Solr logs should tell you something about wh

slave is getting full synced every polling

2016-02-11 Thread Novin Novin
Hi Guys, I'm having a problem with master slave syncing. So I have two cores one is small core (just keep data use frequently for fast results) and another is big core (for rare query and for search in every thing). both core has same solrconfig file. But small core replication is fine, other tha