Sending 32K docs at a time is a bit of overkill, I usually stay around 1,000.
I agree with Michael, it's rarely a Good Thing to do the commit in line, except (perhaps) at the very end. Just let the autocommit settings take care of it for you. Your log shows numbers of "overlapping ondeck searchers" which indicate that you are somehow committing too often and/or your autowarming process is too long somehow. Your solronfig settings would be interesting here. Your server commit is flushing to disk, but then returning before the new searcher is warmed, so I'm guessing that's what's driving the too many on-deck searcher error. All that said, you should be able to open afterwards. So how are you shutting down Solr? Killing it in Eclipse? And what do your logs show when you try to start it back up? There should be some information there. Best, Erick On Wed, May 21, 2014 at 7:59 AM, Michael Della Bitta <michael.della.bi...@appinions.com> wrote: > Two possibly unrelated things: > > 1. Don't commit until the end. > > 2. Consider not optimizing at all. > > You might want to look at your autocommit settings in your solrconfig.xml. > You probably want soft commits set at something north of 10 seconds, and > hard commits set to openSearcher=false with a maxTime somewhat larger than > your soft commit setting, somewhere in the low minutes range. > > > Michael Della Bitta > > Applications Developer > > o: +1 646 532 3062 > > appinions inc. > > “The Science of Influence Marketing” > > 18 East 41st Street > > New York, NY 10017 > > t: @appinions <https://twitter.com/Appinions> | g+: > plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts> > w: appinions.com <http://www.appinions.com/> > > > On Wed, May 21, 2014 at 9:51 AM, Cam Bazz <camb...@gmail.com> wrote: > >> Hello, >> >> I am indexing some 20 million documents in a solr instance. >> >> After i do the indexing, and shut the instance down and back on, it will >> not respond to queries, >> and it will not show collection stats. >> >> I am attaching the solr log to this email. >> >> The way that I do indexing is: >> >> // document list >> Collection<SolrInputDocument> docs = new ArrayList<>(); >> >> // process files >> for(;;) { >> >> // create document and add it to docs list >> if(linenum%(256*128)==0) { >> // every once in a while add them to server >> try { >> server.add(docs); >> server.commit(true, false, false); >> docs.clear(); >> } catch (SolrServerException ex) { >> >> Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex); >> } >> } >> } >> >> // and at the end i optimize >> try { >> server.optimize(true, false); >> } catch (SolrServerException ex) { >> >> Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex); >> } >> >> >> I am suspecting this has to do something with background merges, etc. >> >> Any ideas/help/recomendations on this problem is greatly appreciated. I am >> using solr4.8 >> >> Best Regards, >> C.B. >> >> >>
