On 3 January 2013 05:55, Mark Miller <markrmil...@gmail.com> wrote: > > 32 cores eh? You probably have to raise some limits to take advantage of > that. > > https://issues.apache.org/jira/browse/SOLR-4078 > support configuring IndexWriter max thread count in solrconfig > > That's coming in 4.1 and is likely important - the default is only 8. > > You might always want to experiment with using more merge threads? I think > the default may be 3. > > Beyond that, you may want to look at running multiple jvms on the one host > and doing distributed. That can certainly have benefits, but you have to > weigh against the management costs. And make sure process->processor > affinity is in gear. > > Finally, make sure you are using many threads to add docs... [...]
Yes, making sure to use many threads is definitely good. We also found that indexing to multiple Solr cores, and doing one merge of all the indices at the end dramatically improved indexing time. As long as we had roughly one CPU core per Solr core (I am guessing that had to do with threading) indexing speed increased linearly with the number of Solr cores. Yes, the merge at the end is slow, and needs large disk space (at least twice the total index size), but one wins overall. Regards, Gora
