Mark Miller <markrmil...@gmail.com> wrote on 01/26/2009 04:30:00 PM:
> Just a point or I missed: with such a large index (not doc size large,
> but content wise), I imagine a lot of your 16GB of RAM is being used by
> the system disk cache - which is good. Another reason you don't want to
> give too much RAM to the JVM. But you still want to give it enough to
> avoid the OOM :) Assuming you are using the RAM you are legitimately.
> And I don't yet have a reason to think you are not.
I've bumped the JVM max memory to 2G. Hopefully that is enough. I'll be
keeping an eye on it.
> Also, there has been a report of or two of a lockup that didn't appear
> to involve an OOM, so this is not guaranteed to solve that. However,
> seeing that the lockup comes after the OOM, its the likely first thing
> to fix. Once the memory problems are taken care of, the locking issue
> can be addressed if you find it still remains. My bet is that fixing the
> OOM will clear it up.
I've gone through my code looking for possible leaks and didn't find
anything. That doesn't mean they're not there of course.
I ran an analyzer on the heap dump from the last OOM event. These were the
likely items it identified:
org/apache/catalina/connector/Connector java/util/WeakHashMap
$Entry 399,913,269 bytes
org/apache/catalina/connector/Connector java/lang/Object[ ]
197,256,078 bytes
org/apache/lucene/search/ExtendedFieldCache java/util/WeakHashMap$Entry
[ ] 177,893,021 bytes
org/apache/lucene/search/ExtendedFieldCache java/util/HashMap$Entry[ ]
42,490,720 bytes
org/apache/lucene/search/ExtendedFieldCache java/util/HashMap$Entry[ ]
42,490,656 bytes
I'm not sure what to make of this, though.
> > You also might lower the max warming searchers setting if that makes
> > sense.
I'm using the default setting of 2. I have seen an error about too many
warming searchers once or twice, but not often.
Thanks,
Jerry
