Hello, Emir, Thanks for the reply, this is the solr version and heap info, standalone single solr server. I don't have monitor tool connected. only look at 'top', has not seen cpu spike so far, when the slow response happens, cpu usage is not high at all, around 30%.
# curl 'http://.../solr/admin/info/system?wt=json&indent=true' { "responseHeader":{ "status":0, "QTime":27}, "mode":"std", "solr_home":"/ardome/solr", "lucene":{ "solr-spec-version":"6.5.1", "solr-impl-version":"6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:23:42", "lucene-spec-version":"6.5.1", "lucene-impl-version":"6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15"}, "jvm":{ "version":"1.8.0_144 25.144-b01", "name":"Oracle Corporation Java HotSpot(TM) 64-Bit Server VM", "spec":{ "vendor":"Oracle Corporation", "name":"Java Platform API Specification", "version":"1.8"}, "jre":{ "vendor":"Oracle Corporation", "version":"1.8.0_144"}, "vm":{ "vendor":"Oracle Corporation", "name":"Java HotSpot(TM) 64-Bit Server VM", "version":"25.144-b01"}, "processors":32, "memory":{ "free":"69.1 GB", "total":"180.2 GB", "max":"266.7 GB", "used":"111 GB (%41.6)", "raw":{ "free":74238728336, "total":193470136320, "max":286331502592, "used":119231407984, "used%":41.64103736566334}}, "jmx":{ "bootclasspath":"/usr/java/jdk1.8.0_144/jre/lib/resources.jar:/usr/java/jdk1.8.0_144/jre/lib/rt.jar:/usr/java/jdk1.8.0_144/jre/lib/sunrsasign.jar:/usr/java/jdk1.8.0_144/jre/lib/jsse.jar:/usr/java/jdk1.8.0_144/jre/lib/jce.jar:/usr/java/jdk1.8.0_144/jre/lib/charsets.jar:/usr/java/jdk1.8.0_144/jre/lib/jfr.jar:/usr/java/jdk1.8.0_144/jre/classes", "classpath":"...", "commandLineArgs":["-Xms100G", "-Xmx300G", "-DSTOP.PORT=8079", "-DSTOP.KEY=..", "-Dsolr.solr.home=..", "-Djetty.port=8983"], "startTime":"2019-03-18T09:35:27.892Z", "upTimeMS":9258422}}, "system":{ "name":"Linux", "arch":"amd64", "availableProcessors":32, "systemLoadAverage":14.72, "version":"3.0.101-311.g08a8a9d-default", "committedVirtualMemorySize":2547960700928, "freePhysicalMemorySize":4530696192, "freeSwapSpaceSize":3486846976, "processCpuLoad":0.3257436126790475, "processCpuTime":93869450000000, "systemCpuLoad":0.3279781055816521, "totalPhysicalMemorySize":406480175104, "totalSwapSpaceSize":4302303232, "maxFileDescriptorCount":32768, "openFileDescriptorCount":385, "uname":"Linux ... 3.0.101-311.g08a8a9d-default #1 SMP Wed Dec 14 10:15:37 UTC 2016 (08a8a9d) x86_64 x86_64 x86_64 GNU/Linux\n", "uptime":" 13:09pm up 5 days 21:23, 7 users, load average: 14.72, 12.28, 11.48\n"}} ________________________________ From: Emir Arnautović <emir.arnauto...@sematext.com> Sent: Monday, March 18, 2019 12:10:30 PM To: solr-user@lucene.apache.org Subject: Re: Solr index slow response Hi Aaron, Which version of Solr? How did you configure your heap? Is it standalone Solr or SolrCloud? A single server? Do you use some monitoring tool? Do you see some spikes, pauses or CPU usage is constant? Thanks, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 18 Mar 2019, at 11:47, Aaron Yingcai Sun <y...@vizrt.com> wrote: > > Hello, Solr! > > > We are having some performance issue when try to send documents for solr to > index. The repose time is very slow and unpredictable some time. > > > Solr server is running on a quit powerful server, 32 cpus, 400GB RAM, while > 300 GB is reserved for solr, while this happening, cpu usage is around 30%, > mem usage is 34%. io also look ok according to iotop. SSD disk. > > > Our application send 100 documents to solr per request, json encoded. the > size is around 5M each time. some times the response time is under 1 seconds, > some times could be 300 seconds, the slow response happens very often. > > > "Soft AutoCommit: disabled", "Hard AutoCommit: if uncommited for 3600000ms; > if 1000000 uncommited docs" > > > There are around 100 clients sending those documents at the same time, but > each for the client is blocking call which wait the http response then send > the next one. > > > I tried to make the number of documents smaller in one request, such as 20, > but still I see slow response time to time, like 80 seconds. > > > Would you help to give some hint how improve the response time? solr does > not seems very loaded, there must be a way to make the response faster. > > > BRs > > //Aaron > > >
