Re: SolrCloud shard splitting keeps failing

2013-10-08 Thread Shalin Shekhar Mangar
I was wrong in saying that we don't need to open a searcher, we do. I committed a fix in SOLR-5314 to use soft commits instead of hard commits. I also increased the read time out value. Both of these together will reduce the likelyhood of such a thing happening. https://issues.apache.org/jira/brow

Re: SolrCloud shard splitting keeps failing

2013-10-08 Thread Shalin Shekhar Mangar
Hi Kalle, The problem here is that certain actions are taking too long causing the split process to terminate in between. For example, a commit on the parent shard leader took 83 seconds in your case but the read timeout value is set to 60 seconds only. We actually do not need to open a searcher d

Re: SolrCloud shard splitting keeps failing

2013-10-08 Thread Harald Kirsch
Hello Kalle, we noticed the same problem some weeks ago: http://lucene.472066.n3.nabble.com/Share-splitting-at-23-million-documents-gt-OOM-td4085064.html Would be interesting to hear if there is more positive feedback this time. We finally concluded that it may be worth to start with many shar

SolrCloud shard splitting keeps failing

2013-10-07 Thread Kalle Aaltonen
I have a test system where I have a index of 15M documents in one shard that I would like to split in two. I've tried it four times now. I have a stand-alone zookeeper running on the same machine. The end result is that I have two new shards with state "construction", and each has one replica whic