On 5-Dec-08, at 2:24 PM, wojtekpia wrote:
I've seen some strangle results in the last few days of testing, but
this one
flies in the face of everything I've read on this forum: Reducing
filterCache size has increased performance.
This isn't really unexpected behaviour. The problem with a huge
filter cache is that it is fighting with OS disk cache--the latter of
which can be much much more important. Reducing the size of the
filter cache give more to the OS.
Try giving 17GB to java, and letting the OS cache the entire index.
Increase the filter cache as much as you can without OOM'ing. That
should give optimal performance. Note that you don't always need the
_whole_ index in the os cache to get acceptable performance, but if
you can afford it, it is a good idea.
It is also possible that you are experiencing contention in the
filtercache code--have you tried the concurrent filter cache impl?
-Mike
I have posted my setup here:
http://www.nabble.com/Throughput-Optimization-td20335132.html.
My original filterCache was 700,000. Reducing it to 20,000, I found:
- Average response time decreased by 85%
- Average throughput increased by 250%
- CPU time used by the garbage collector decreased by 85%
- The system showed to weird GC issues (reported yesterday at:
http://www.nabble.com/new-faceting-algorithm-td20674902.html)
Further reducing the filterCache to 10,000
- Average response time decreased by another 27%
- Average throughput increased by another 30%
- GC CPU usage also dropped
- System behavior changed after ~30 minutes, with a slight performance
degradation
These results came from a load test. I'm running trunk code from Dec
2 with
Yonik's faceting improvement turned on.
Any thoughts?
--
View this message in context:
http://www.nabble.com/Smaller-filterCache-giving-better-performance-tp20863674p20863674.html
Sent from the Solr - User mailing list archive at Nabble.com.
