One more thing - it is considered a good practice to use the same value for Xmx 
and Xms.

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 18 Mar 2019, at 14:19, Emir Arnautović <emir.arnauto...@sematext.com> 
> wrote:
> 
> Hi Aaron,
> Without looking too much into numbers, my bet would be that it is large heap 
> that is causing issues. I would decrease is significantly (<30GB) and see if 
> it is enough for your max load. Also, disable swap or reduce swappiness to 
> min.
> 
> In any case, you should install some monitoring tool that would help you do 
> better analysis when you run into problems. One such tool is our monitoring 
> solution: https://sematext.com/spm
> 
> HTH,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> 
> 
> 
>> On 18 Mar 2019, at 13:14, Aaron Yingcai Sun <y...@vizrt.com> wrote:
>> 
>> Hello, Emir,
>> 
>> Thanks for the reply, this is the solr version and heap info, standalone 
>> single solr server. I don't have monitor tool connected. only look at 'top', 
>> has not seen cpu spike so far, when the slow response happens, cpu usage is 
>> not high at all, around 30%.
>> 
>> 
>> # curl 'http://.../solr/admin/info/system?wt=json&indent=true'
>> {
>> "responseHeader":{
>>   "status":0,
>>   "QTime":27},
>> "mode":"std",
>> "solr_home":"/ardome/solr",
>> "lucene":{
>>   "solr-spec-version":"6.5.1",
>>   "solr-impl-version":"6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - 
>> jimczi - 2017-04-21 12:23:42",
>>   "lucene-spec-version":"6.5.1",
>>   "lucene-impl-version":"6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - 
>> jimczi - 2017-04-21 12:17:15"},
>> "jvm":{
>>   "version":"1.8.0_144 25.144-b01",
>>   "name":"Oracle Corporation Java HotSpot(TM) 64-Bit Server VM",
>>   "spec":{
>>     "vendor":"Oracle Corporation",
>>     "name":"Java Platform API Specification",
>>     "version":"1.8"},
>>   "jre":{
>>     "vendor":"Oracle Corporation",
>>     "version":"1.8.0_144"},
>>   "vm":{
>>     "vendor":"Oracle Corporation",
>>     "name":"Java HotSpot(TM) 64-Bit Server VM",
>>     "version":"25.144-b01"},
>>   "processors":32,
>>   "memory":{
>>     "free":"69.1 GB",
>>     "total":"180.2 GB",
>>     "max":"266.7 GB",
>>     "used":"111 GB (%41.6)",
>>     "raw":{
>>       "free":74238728336,
>>       "total":193470136320,
>>       "max":286331502592,
>>       "used":119231407984,
>>       "used%":41.64103736566334}},
>>   "jmx":{
>>     
>> "bootclasspath":"/usr/java/jdk1.8.0_144/jre/lib/resources.jar:/usr/java/jdk1.8.0_144/jre/lib/rt.jar:/usr/java/jdk1.8.0_144/jre/lib/sunrsasign.jar:/usr/java/jdk1.8.0_144/jre/lib/jsse.jar:/usr/java/jdk1.8.0_144/jre/lib/jce.jar:/usr/java/jdk1.8.0_144/jre/lib/charsets.jar:/usr/java/jdk1.8.0_144/jre/lib/jfr.jar:/usr/java/jdk1.8.0_144/jre/classes",
>>     "classpath":"...",
>>     "commandLineArgs":["-Xms100G",
>>       "-Xmx300G",
>>       "-DSTOP.PORT=8079",
>>       "-DSTOP.KEY=..",
>>       "-Dsolr.solr.home=..",
>>       "-Djetty.port=8983"],
>>     "startTime":"2019-03-18T09:35:27.892Z",
>>     "upTimeMS":9258422}},
>> "system":{
>>   "name":"Linux",
>>   "arch":"amd64",
>>   "availableProcessors":32,
>>   "systemLoadAverage":14.72,
>>   "version":"3.0.101-311.g08a8a9d-default",
>>   "committedVirtualMemorySize":2547960700928,
>>   "freePhysicalMemorySize":4530696192,
>>   "freeSwapSpaceSize":3486846976,
>>   "processCpuLoad":0.3257436126790475,
>>   "processCpuTime":93869450000000,
>>   "systemCpuLoad":0.3279781055816521,
>>   "totalPhysicalMemorySize":406480175104,
>>   "totalSwapSpaceSize":4302303232,
>>   "maxFileDescriptorCount":32768,
>>   "openFileDescriptorCount":385,
>>   "uname":"Linux ... 3.0.101-311.g08a8a9d-default #1 SMP Wed Dec 14 10:15:37 
>> UTC 2016 (08a8a9d) x86_64 x86_64 x86_64 GNU/Linux\n",
>>   "uptime":" 13:09pm  up 5 days 21:23,  7 users,  load average: 14.72, 
>> 12.28, 11.48\n"}}
>> 
>> 
>> 
>> 
>> ________________________________
>> From: Emir Arnautović <emir.arnauto...@sematext.com>
>> Sent: Monday, March 18, 2019 12:10:30 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Solr index slow response
>> 
>> Hi Aaron,
>> Which version of Solr? How did you configure your heap? Is it standalone 
>> Solr or SolrCloud? A single server? Do you use some monitoring tool? Do you 
>> see some spikes, pauses or CPU usage is constant?
>> 
>> Thanks,
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>> 
>> 
>> 
>>> On 18 Mar 2019, at 11:47, Aaron Yingcai Sun <y...@vizrt.com> wrote:
>>> 
>>> Hello, Solr!
>>> 
>>> 
>>> We are having some performance issue when try to send documents for solr to 
>>> index. The repose time is very slow and unpredictable some time.
>>> 
>>> 
>>> Solr server is running on a quit powerful server, 32 cpus, 400GB RAM, while 
>>> 300 GB is reserved for solr, while this happening, cpu usage is around 30%, 
>>> mem usage is 34%.  io also look ok according to iotop. SSD disk.
>>> 
>>> 
>>> Our application send 100 documents to solr per request, json encoded. the 
>>> size is around 5M each time. some times the response time is under 1 
>>> seconds, some times could be 300 seconds, the slow response happens very 
>>> often.
>>> 
>>> 
>>> "Soft AutoCommit: disabled", "Hard AutoCommit: if uncommited for 3600000ms; 
>>> if 1000000 uncommited docs"
>>> 
>>> 
>>> There are around 100 clients sending those documents at the same time, but 
>>> each for the client is blocking call which wait the http response then send 
>>> the next one.
>>> 
>>> 
>>> I tried to make the number of documents smaller in one request, such as 20, 
>>> but  still I see slow response time to time, like 80 seconds.
>>> 
>>> 
>>> Would you help to give some hint how improve the response time?  solr does 
>>> not seems very loaded, there must be a way to make the response faster.
>>> 
>>> 
>>> BRs
>>> 
>>> //Aaron
>>> 
>>> 
>>> 
>> 
> 

Reply via email to