Re: Slave index size growing fast

2012-03-26 Thread Alexandre Rocco
Erick, I haven't changed the maxCommitsToKeep yet. We stopped the slave that had issues and removed the data dir as you pointed and afer starting it, everything started working as normal. I guess that at some point someone commited on the slave or even copied the master files over and made this me

Re: Slave index size growing fast

2012-03-23 Thread Erick Erickson
Alexandre: Have you changed anything like on your slave? And do you have more than one slave? If you do, have you considered just blowing away the entire .../data directory on the slave and letting it re-start from scratch? I'd take the slave out of service for the duration of this operation, or

Re: Slave index size growing fast

2012-03-23 Thread Alexandre Rocco
Tomás, The 300+GB size is only inside the index.20110926152410 dir. Inside there are a lot of files. I am almost conviced that something is messed up like someone commited on this slave machine. Thanks 2012/3/23 Tomás Fernández Löbbe > Alexandre, additionally to what Erick said, you may want t

Re: Slave index size growing fast

2012-03-23 Thread Alexandre Rocco
Erick, The master /data dir contains only an index dir with a bunch of files. In the slave, the /data dir contains an index.20110926152410 dir with a lot more files than the master. That is quite strange for me. I guess that the config is right, since we have another slave that is running fine wi

Re: Slave index size growing fast

2012-03-23 Thread Tomás Fernández Löbbe
Alexandre, additionally to what Erick said, you may want to check in the slave if what's 300+GB is the "data" directory or the "index." directory. On Fri, Mar 23, 2012 at 12:25 PM, Erick Erickson wrote: > not really, unless perhaps you're issuing commits or optimizes > on the _slave_ (which you s

Re: Slave index size growing fast

2012-03-23 Thread Erick Erickson
not really, unless perhaps you're issuing commits or optimizes on the _slave_ (which you should NOT do). Replication happens based on the version of the index on the master. True, it starts out as a timestamp, but then successive versions just have that number incremented. The version number in th

Re: Slave index size growing fast

2012-03-23 Thread Alexandre Rocco
Erick, We're using Solr 3.3 on Linux (CentOS 5.6). The /data dir on master is actually 1.2G. I haven't tried to recreate the index yet. Since it's a production environment, I guess that I can stop replication and indexing and then recreate the master index to see if it makes any difference. Also

Re: Slave index size growing fast

2012-03-23 Thread Erick Erickson
What version of Solr and what operating system? But regardless, this shouldn't be happening. Indexes can temporarily double in size, but any extras should be cleaned up relatively soon. On the master, what's the total size of the /data directory? I'm a little suspicious of the on your master, bu

Slave index size growing fast

2012-03-23 Thread Alexandre Rocco
Hello, We have a Solr index that has an average of 1.19 GB in size. After configuring the replication, the slave machine is growing the index size expoentially. Currently we have an slave with 323.44 GB in size. Is there anything that could cause this behavior? The current replication config is be