I have a question for developers of Solr regarding the issue of "left-over" index folders when replication fails. Could be this issue resolved quickly if when replication starts Solr creates a "flag file" in "index.XXXX" folder and when replication ends (and commits) this file is deleted? In this case if a server is restarted (or on schedule) it could quickly scan all the "index.XXXX" folders and delete those (maybe not the last one or those relevant to the index.properties file) that still *contain* a flag file and are so unfinished and uncommited.
I have not really looked at the code yet so I may have a different view on the workings of replication. Would the solution I described at least address this issue? Best regards, Primoz From: primoz.sk...@policija.si To: solr-user@lucene.apache.org Date: 11.10.2013 12:46 Subject: Re: Cores with lot of folders with prefix index.XXXXXXX Thanks, I guess I was wrong after all in my last post. Primož From: Shalin Shekhar Mangar <shalinman...@gmail.com> To: solr-user@lucene.apache.org Date: 11.10.2013 12:43 Subject: Re: Cores with lot of folders with prefix index.XXXXXXX There are open issues related to extra index.XXX folders lying around if replication/recovery fails. See https://issues.apache.org/jira/browse/SOLR-4506 On Fri, Oct 11, 2013 at 4:06 PM, Yago Riveiro <yago.rive...@gmail.com>wrote: > The thread that you point is about master / slave - replication, Is this > issue valid on SolrCloud context? > > I check the index.properties and indeed the variable index=index.XXXXX > point to a folder, the others can be deleted without any scary side effect? > > > -- > Yago Riveiro > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Friday, October 11, 2013 at 11:31 AM, primoz.sk...@policija.si wrote: > > > Do you have a lot of failed replications? Maybe those folders have > > something to do with this (please see the last answer at > > > http://stackoverflow.com/questions/3145192/why-does-my-solr-slave-index-keep-growing > > ). If your disk space is valuable check index.properties file under data > > folder and try to determine which folders can be safely deleted. > > > > Primo¾ > > > > > > > > > > From: Yago Riveiro <yago.rive...@gmail.com (mailto: > yago.rive...@gmail.com)> > > To: solr-user@lucene.apache.org (mailto:solr-user@lucene.apache.org) > > Date: 11.10.2013 12:13 > > Subject: Re: Cores with lot of folders with prefix index.XXXXXXX > > > > > > > > I have ssd's therefor my space is like gold, I can have 30% of my space > > waste in failed replications, or replications that are not cleaned. > > > > The question for me is if this a normal behaviour or is a bug. If is a > > normal behaviour I have a trouble because a ssd with more than 512G is > > expensive. > > > > -- > > Yago Riveiro > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > On Friday, October 11, 2013 at 11:03 AM, primoz.sk...@policija.si(mailto: > primoz.sk...@policija.si) wrote: > > > > > I think this is connected to replications being made? I also have quite > > > some of them but currently I am not worried :) > > > > > > > > > > > > -- Regards, Shalin Shekhar Mangar.
