On 4/17/2015 8:14 PM, Kamal Kishore Aggarwal wrote:
Hi,
As per this article, the linux machine is preferred to have 1.5 times RAM
with respect to index size. So, to verify this, I tried testing the solr
performance in different volumes of RAM allocation keeping other
configuration (i.e Solid State Drives, 8 core processor, 64-Bit) to be same
in both the cases. I am using solr 4.8.1 with tomcat server.
https://wiki.apache.org/solr/SolrPerformanceProblems
1) Initially, the linux machine had 32 GB RAM, out of which I allocated
14GB to solr.
export CATALINA_OPTS="-Xms2048m -Xmx14336m -XX:+UseConcMarkSweepGC
-XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -Xloggc:./logs/info_error/tomcat_gcdetails.log"
The average search time for 1000 queries 300ms.
2) After that, RAM was increased to 68 GB, out of which I allocated 40GB to
Solr. Now, on a strange note, the average search time for the same set of
queries was 3000ms.
Now, after this, I reduced solr allocated RAM to 25GB on 68GB machine. But,
still the search time was higher as compared to first case.
What am I missing. Please suggest.
How much disk space do all your indexes on that Solr server use?
The discrepancy in query time is almost guaranteed to be because of
garbage collection pauses. There is no need to increase the Solr heap
size ... if you had kept the heap at 14GB while increasing the RAM, the
average qtime would have either stayed the same or decreased.
GC tuning is required for large heaps in Solr, which typically means
anything over about 2GB. You have enabled the concurrent mark sweep
collector, but you have no other tuning. I bet that if you tune your
GC, you can get a lot better than 300ms average.
On the following wiki page, I have condensed all the knowledge I have
about GC tuning for Solr:
http://wiki.apache.org/solr/ShawnHeisey
The "bin/solr" script that starts Solr 5.x uses a heavily tuned CMS
config, very similar to the CMS config that's on my wiki page.
Here's some more info about Solr performance with respect to RAM:
http://wiki.apache.org/solr/SolrPerformanceProblems
Thanks,
Shawn
