Re: Solr+HDFS

2016-02-05 Thread Erick Erickson
bq: I assume this would go along with also increasing autoCommit? Not necessarily, the two are have much different consequences if openSearcher is set to false for autoCommit. Essentially all this is doing is flushing the current segments to disk and opening new segments, no autowarming etc. is be

Re: Solr+HDFS

2016-02-05 Thread Joseph Obernberger
Thank you Shawn. Sounds like increasing the autoSoftCommit maxTime would be a good idea. I assume this would go along with also increasing autoCommit? All of our collections (just 2 at the moment) have the same settings. The data directory is in HDFS and is the same data directory for every shar

Re: Solr+HDFS

2016-02-05 Thread Shawn Heisey
On 2/5/2016 8:11 AM, Joseph Obernberger wrote: > Thank you for the reply Scott - we have the commit settings as: > > 6 > false > > > 15000 > > > Is that 50% disk space rule across the entire HDFS cluster or on an > individual spindle? That autoSoftCommit maxTime is pret

Re: Solr+HDFS

2016-02-05 Thread Joseph Obernberger
I'm wondering if the shutdown time is too short. When we shutdown the cluster, could it be that it doesn't have enough time to flush? It only happens some of the time; as to which node is seems to be random. -Joe On Tue, Feb 2, 2016 at 12:49 PM, Erick Erickson wrote: > Does this happen all th

Re: Solr+HDFS

2016-02-05 Thread Joseph Obernberger
Thank you for the reply Scott - we have the commit settings as: 6 false 15000 Is that 50% disk space rule across the entire HDFS cluster or on an individual spindle? Thank you! -Joe On Tue, Feb 2, 2016 at 12:01 PM, Scott Stults < sstu...@opensourceconnections.com> wr

Re: Solr+HDFS

2016-02-02 Thread Erick Erickson
Does this happen all the time or only when bringing up Solr on some of the nodes? My (naive) question is whether this message: AlreadyBeingCreatedException could indicate that more than one Solr is trying to access the same tlog Best, Erick On Tue, Feb 2, 2016 at 9:01 AM, Scott Stults wrote

Re: Solr+HDFS

2016-02-02 Thread Scott Stults
It seems odd that the tlog files are so large. HDFS aside, is there a reason why you're not committing? Also, as far as disk space goes, if you dip below 50% free you run the risk that the index segments can't be merged. k/r, Scott On Fri, Jan 29, 2016 at 12:40 AM, Joseph Obernberger < joseph.ob

Re: Solr + HDFS settings

2014-10-27 Thread Norgorn
Already tried with same result (the message changed properly ) -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-HDFS-settings-tp4165873p4166089.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr + HDFS settings

2014-10-27 Thread Michael Della Bitta
This doesn't answer your question, but unless something is changed, you're going to want to set this to false. It causes index corruption at the moment. On 10/25/14 03:42, Norgorn wrote: true

Re: Solr + HDFS settings

2014-10-25 Thread Norgorn
Ok, new problem, while collection or shard creating: Caused by: no segments* file found in NRTCachingDirectory(HdfsDirectory@3a19dc74 lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@43507d1b; maxCacheMB=192.0 maxMergeSizeMB=16.0): files: [HdfsDirectory@3a19dc74 lockFactory=org.apache.solr.s