Re: slave index is bigger than master index

2010-07-29 Thread Peter Karich
Hi Muneeb, I fear you'll have no chance: replicating an index will use more disc space on the slave nodes. Of course, you could minimize disc usage AFTER the replication via the 'optimize-hack'. But are you sure the reason for the slave-node die, is due to disc limitations? Try to observe the sla

Re: slave index is bigger than master index

2010-07-29 Thread Muneeb Ali
Where do these lines go in solr config? 5000 1 Thanks, -Mueeb -- View this message in context: http://lucene.472066.n3.nabble.com/slave-index-is-bigger-than-master-index-tp996329p1003903.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: slave index is bigger than master index

2010-07-29 Thread Muneeb Ali
Well I do have disk limitations too, and thats why I think slave nodes died, when replicating data from master node. (as it was just adding on top of existing index files). What do you mean here? Optimizing is too CPU expensive? What I meant by avoid playing around with slave nodes is that doing

Re: slave index is bigger than master index

2010-07-28 Thread Muneeb Ali
>> In solrconfig.xml, these two lines control that. Maybe they need to be increased. >> 5000 >> 1 Where do I add those in solrconfig? These lines doesn't seem to be present in the example solrconfig file... -- View this message in context: http://lucene.472066.n3.nabble.com/slave-i

Re: slave index is bigger than master index

2010-07-28 Thread Muneeb Ali
Well I do have disk limitations too, and thats why I think slave nodes died, when replicating data from master node. (as it was just adding on top of existing index files). :: What do you mean here? Optimizing is too CPU expensive? What I meant by avoid playing around with slave nodes is that d

Re: slave index is bigger than master index

2010-07-27 Thread Lance Norskog
Ah! You have junk files piling up in the slave index directory. When this happens, you may have to remove data/index entirely. I'm not sure if Solr replication will handle that, or if you have to copy the whole index to reset it. You said the slaves time out- maybe the files are so large that the

Re: slave index is bigger than master index

2010-07-27 Thread Peter Karich
> We have three dedicated servers for solr, two for slaves and one for master, > all with linux/debian packages installed. > > I understand that replication does always copies over the index in an exact > form as in master index directory (or it is supposed to do that at least), > and if the mast

Re: slave index is bigger than master index

2010-07-27 Thread Muneeb Ali
We have three dedicated servers for solr, two for slaves and one for master, all with linux/debian packages installed. I understand that replication does always copies over the index in an exact form as in master index directory (or it is supposed to do that at least), and if the master index wa

Re: slave index is bigger than master index

2010-07-26 Thread Chris Hostetter
: No I didn't. I thought you aren't supposed to run optimize on slaves. Well correct, you should make all changes to the master. : but it doesn;t matter now, as I think its fixed now. I just added a dummy : document on master, ran a commit call and then once that executed ran an : optimize call.

RE: slave index is bigger than master index

2010-07-26 Thread Bastian Spitzer
? maybe worth a try.. good luck -Ursprüngliche Nachricht- Von: Peter Karich [mailto:peat...@yahoo.de] Gesendet: Montag, 26. Juli 2010 16:54 An: solr-user@lucene.apache.org Betreff: Re: slave index is bigger than master index did you try an optimize on the slave too? > Yes I always run

Re: slave index is bigger than master index

2010-07-26 Thread Muneeb Ali
No I didn't. I thought you aren't supposed to run optimize on slaves. Well but it doesn;t matter now, as I think its fixed now. I just added a dummy document on master, ran a commit call and then once that executed ran an optimize call. This triggered snapshooter to replicate the index, which some

Re: slave index is bigger than master index

2010-07-26 Thread Peter Karich
did you try an optimize on the slave too? > Yes I always run an optimize whenever I index on master. In fact I just ran > an optimize command an hour ago, but it didn't make any difference. >

Re: slave index is bigger than master index

2010-07-26 Thread Muneeb Ali
I just checked my config file, and I do have exact same values for deletionPolicy tag, as you attached in your email, so I dont really think it could be this. -- View this message in context: http://lucene.472066.n3.nabble.com/slave-index-is-bigger-than-master-index-tp996329p996373.html Sent fr

Re: slave index is bigger than master index

2010-07-26 Thread Tommaso Teofili
Hi, I think that you may be using a Lucene/Solr IndexDeletionPolicy that does not remove old commits (and you aren't propagating solr-config via replication). You can configre this feature on the solr-config.xml inside the tag: * 1 0 * I hope this can be help