On 8 March 2012 16:18, gabriel shen <xshco...@gmail.com> wrote: > Our indexing process is to adding a bundle of solr documents(for example > 5000) to solr each time, and we observed that before commiting(which might > be io bounded) it uses less than half the CPU capacity constantly, which > sounds strange to us why it doesn't use full cpu power. As for RAM, I don't > know how much it will affect CPU utilization, we have assigned 14gb to the > solr tomcat server on a 32 gb linux machine. [...]
Are you hitting memory limits? As Tanguy has already pointed out in nice detail, it probably also does matter how you push documents to Solr, and how often you commit. In an apples-to-oranges comparison, we used to run a large indexing task, but with only a single commit at the end, while it sounds as if you are using smaller batches, with more frequent commits. In our case, we could max out CPU usage (well, we backed off at ~85% utilisation on each core). Though we were fetching data over the network, it was a relatively high-bandwidth internal connection, and we were using DIH with multiple Solr cores. Regards, Gora
