On 12/13/2016 6:09 AM, forest_soup wrote: > I posted this issue to a JIRA. Could anyone help comment? Thanks! > > https://issues.apache.org/jira/browse/SOLR-9741
Please use the mailing list *before* opening an issue in Jira. If at all possible, we want to be sure that problems are caused by a real bug in the software before an issue is created. > When we doing a batch of index and search operations to SolrCloud v5.3.2, we > usually met a CPU% spike lasting about 10 min. > We have 5 physical servers, 2 solr instances running on each server with > different port(8983 and 8984), all 8983 are in a same solrcloud, all 8984 > are in another solrcloud. > > You can see the chart in the attach file screenshot-1.png. > <http://lucene.472066.n3.nabble.com/file/n4309529/screenshot-1.png> > > The thread dump are in the attach file threads.zip. > threads.zip <http://lucene.472066.n3.nabble.com/file/n4309529/threads.zip> > > During the spike, the thread dump shows most of the threads are with the > call stacks below: That stacktrace indicates the thread is doing a query. If most of the threads have that stacktrace, it means Solr is handling a lot of simultaneous queries. That can cause a CPU spike. I checked one of the thread dumps Indexing tends to use a lot of resources. If you are doing all your indexing to the same HTTP endpoint (in a way that doesn't send the request to the correct shard leader), that will also make Solr work harder. You appear to be running Solr with SSL. This is going to increase CPU requirements. I wouldn't expect the increase to be very high, but if CPU is already a problem, that will make it worse. Your iowait CPU percentage appears to be nearly nonexistent, so I might be barking up the wrong tree with some of the following questions, but I'll go ahead and ask them anyway: * What is the total physical memory in the machine? * What is the max heap on each of the two Solr processes? * What is the total index size in each Solr process? * What is the total tlog size in each Solr process? * What are your commit characteristics like -- both manual and automatic. * Do you have WARN or ERROR messages in your logfile? Thanks, Shawn
