On 2/8/2018 12:41 AM, rubi.hali wrote:
> We are using Solr-6.6.2 with one master and 4 Slaves each having 4 CPU core
> and 16GB RAM. We were doing load testing with 4000 users and 800 odd search
> keywords which resulted into 95% of CPU usage in less than 3 minutes and
> affected our QUERY Responses. There was spike in physical memory also which
> was not going down even when we stopped sending load.
How many queries per second is that sending? Thousands of queries per
second may need more than five servers.
> Our JVM Heap given to Solr is 8G which still lefts with 8G for OS.
>
> We have 4 lakh documents in solr.
How big (disk space) is all the index data being handled by one of your
Solr servers? Does all that index data add up to 400000 documents, or
is that just the document count for one index out of multiple?
> Our Cache Configurations done in SOLR are
>
> <documentCache class="solr.LRUCache"
> size="50000"
> initialSize="30000"
> autowarmCount="0"/>
This is fairly large. May not be a problem, but that will depend on how
big each document is.
> <filterCache class="solr.FastLRUCache"
> size="4096"
> initialSize="512"
> autowarmCount="20"/>
This is quite large for a filterCache. It probably needs to be
smaller. Be careful with autowarming on the filterCache -- sometimes
filters can execute very slowly. I had to reduce my autowarmCount on
the filterCache to *four* in order for commits to be fast enough.
> <queryResultCache class="solr.LRUCache"
> size="256"
> initialSize="256"
> autowarmCount="0"/>
I would probably use a non-zero autowarmCount here. But don't make it
TOO large.
> We have enabled autocommit
> <autoCommit>
> <maxDocs>${solr.autoCommit.maxDocs:25000}</maxDocs>
> <maxTime>${solr.autoCommit.maxTime:60000}</maxTime>
> <openSearcher>true</openSearcher> (This is true only in case of
> slaves)
> </autoCommit>
I would recommend that you remove maxDocs and let it just be
time-based. Also, *all* servers should have openSearcher set to false
on autoCommit.
> We are also doing softcommit
> <autoSoftCommit>
> <maxTime>${solr.autoSoftCommit.maxTime:300000}</maxTime>
> </autoSoftCommit>
This is good. You could probably lower it to two minutes instead of five.
> Our Queries are enabled with Grouping so Query Result Cache doesnt get used.
> But still in heavy load we are seeing this behaviour which is resulting into
> high response times.
>
> Please suggest if there is any configuration mismatch or os issue which we
> should resolve for bringing down our High Response Times
If your query load is going to be high, you probably need to add more
servers.
To check whether things are optimized well on each server: What OS is
it running on? I would like to get some more information from your
install, but the exact method for obtaining that information will vary
depending on the OS.
Can you also provide a large solr_gc.log file so that can be analyzed?
Thanks,
Shawn
