Two possibly unrelated things: 1. Don't commit until the end.
2. Consider not optimizing at all. You might want to look at your autocommit settings in your solrconfig.xml. You probably want soft commits set at something north of 10 seconds, and hard commits set to openSearcher=false with a maxTime somewhat larger than your soft commit setting, somewhere in the low minutes range. Michael Della Bitta Applications Developer o: +1 646 532 3062 appinions inc. “The Science of Influence Marketing” 18 East 41st Street New York, NY 10017 t: @appinions <https://twitter.com/Appinions> | g+: plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts> w: appinions.com <http://www.appinions.com/> On Wed, May 21, 2014 at 9:51 AM, Cam Bazz <camb...@gmail.com> wrote: > Hello, > > I am indexing some 20 million documents in a solr instance. > > After i do the indexing, and shut the instance down and back on, it will > not respond to queries, > and it will not show collection stats. > > I am attaching the solr log to this email. > > The way that I do indexing is: > > // document list > Collection<SolrInputDocument> docs = new ArrayList<>(); > > // process files > for(;;) { > > // create document and add it to docs list > if(linenum%(256*128)==0) { > // every once in a while add them to server > try { > server.add(docs); > server.commit(true, false, false); > docs.clear(); > } catch (SolrServerException ex) { > > Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex); > } > } > } > > // and at the end i optimize > try { > server.optimize(true, false); > } catch (SolrServerException ex) { > > Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex); > } > > > I am suspecting this has to do something with background merges, etc. > > Any ideas/help/recomendations on this problem is greatly appreciated. I am > using solr4.8 > > Best Regards, > C.B. > > >
