On 10/7/2011 9:48 AM, Tristan Roddis wrote:
I was advised that, to increase performance, we should increase the
amount of memory allocated to Tomcat so that Solr could use this to
cache query results and so reduce CPU usage and disk I/O. So, I set
both the min and max RAM allocation for Tomcat to 10Gb, which is well
within the amount allocated to the server (16Gb). I tried this on
another machine, set a simulated load of lots of concurrent users
running on our application, and the VisualVM results can be seen at
http://imgur.com/4Om2E&tqLKWl <http://imgur.com/4Om2E&tqLKWl>
These results were very different: in the first half of the graph,
when the system was under load, the amount of allocated heap size was
fixed at 10Gb, which I would expect, but the amount of used heap
climbed gradually to around 3Gb, but then shot right down again to
something close to zero every minute or so, presumably due to the GC
kicking in, giving the sawtooth pattern you can see in the screenshot
above.
My question is: is this normal and expected? I would have thought that
the amount of used heap would climb to somewhere much nearer the
maximum available of 10Gb, rather than peaking at a mere 3Gb. And also
that any garbage collection would only deallocate part of the memory
rather than knocking it right back to zero.
Essentially, why isn't Solr using more of the memory assigned, and
using it constantly rather than continually clearing the cache?
Tristan,
Just assigning more memory to the JVM willl not make Solr utilize it for
caches. You must also increase the cache sizes in solrconfig.xml. That
may not be the best use for your RAM, though. Here are my cache
settings for the cores that handle queries. My shards are nearing 20GB
and have more than 10 million documents in them. I use low
autowarmCount values so that commit times are short.
<filterCache
class="solr.FastLRUCache"
size="64"
initialSize="64"
autowarmCount="4"
cleanupThread="true"
/>
<queryResultCache
class="solr.FastLRUCache"
size="512"
initialSize="512"
autowarmCount="8"
cleanupThread="true"
/>
<documentCache
class="solr.FastLRUCache"
size="16384"
initialSize="4096"
cleanupThread="true"
/>
Although Solr's built in caching does offer performance improvements,
beyond a certain size it won't provide any actual benefit. The best
thing to do (in most cases) is to allocate enough RAM to the JVM to keep
it from ever reaching an out of memory condition, then leave the rest of
it available for OS disk cache. It sounds like 2-3GB might be a good
max heap size for you. If the default tomcat settings did not lead to
any out of memory errors, you might even be able to use 1GB.
Ideally the OS disk cache will be big enough to cache your entire index,
which would mean that during normal operation the system never has to
actually access the disk. That isn't always possible, but you should
get as close to that ideal as you can.
Thanks,
Shawn
