Hi John,
WeakReferences allow things to get GC'd, if there are no other
references to the object referred to.
My understanding is that WeakHashMaps use weak references for the Keys
in the HashMap.
What this means is that the keys in HashMap can be GC'd, once there
are no other references to the key. I _think_ this occurs when the
IndexReader is closed.
It does not mean that objects in the FieldCache will get evicted in
low memory conditions, unless that field cache entry is no longer
needed (i.e. the IndexReader has closed). It just means they can be
collected, when they are no longer needed (but not before).
So, if you are seeing the FieldCache for the current IndexReader
taking up 2.1, that's probably for the current cache usage.
There isn't a "knob" you can turn to cut the cache size, but you can
evaluate your usage of the cache. Some ideas:
How many fields are you searching on? Sorting on? Are you sorting on
String fields, where you could be using a numeric field? Numerics save
space. Do you need to sort on every field that you are sorting on?
Could you facet on fewer fields? For a String field, do you have too
many distinct values? If so, can you reduce the number or unique
terms? You might check your faceting algorithms, and see if you could
use enum, instead of fc for some of them.
Check your statistics page, what's your insanity count?
Tom
On Fri, Dec 10, 2010 at 12:17 PM, John Russell <jjruss...@gmail.com> wrote:
> I have been load testing solr 1.4.1 and have been running into OOM errors.
> Not out of heap but with the GC overhead limit exceeded message meaning that
> it didn't actually run out of heap space but just spent too much CPU time
> trying to make room and gave up.
>
> I got a heap dump and sent it through the Eclipse MAT and found that a
> single WeakHashMap in FieldCacheImpl called readerCache is taking up 2.1GB
> of my 2.6GB heap.
>
> From my understanding of WeakHashMaps the GC should be able to collect those
> references if it needs to but for some reason it isn't here.
>
> My questions are:
>
> 1) Any ideas why the GC is not collecting those weak references in that
> single hashmap?
> 2) Is there a nob in the solr config that can limit the size of that cache?
>
>
> Also, after the OOM is thrown solr doesn't respond much at all and throws
> the exception below, however when I go to the code I see this
>
> try {
> processor.processAdd(addCmd);
> addCmd.clear();
> } catch (IOException e) {
> throw new
> SolrException(SolrException.ErrorCode.SERVER_ERROR, "ERROR adding document "
> + document);
> }
> }
>
> So its swallowing the IOException and throwing a new one without setting
> the cause so I can't see what the IOException is. Is this fixed in any
> newer version? Should I open a bug?
>
>
> Thanks a lot for your help
>
> John
>
>
> SEVERE: org.apache.solr.common.SolrException: ERROR adding document
> SolrInputDocument[{de.id=de.id(1.0)={C2B3B03F1000012C549254560A568C18},
> de.type=de.type(1.0)={Social
> Contact}, sc.author=sc.author(1.0)={Author-3944},
> sc.sourceType=sc.sourceType(1.0)={rss}, sc.link=sc.link(1.0)={
> http://www.cisco.com/feed/date_12.07.10_16.18.03/idx/107
> 52}, sc.title=sc.title(1.0)={Title-erat metus eget vestibulum},
> sc.publishedDate=sc.publishedDate(1.0)={Tue Dec 07 16:22:09 EST 2010},
> sc.createdDate=sc.createdDate(1.0
> )={Tue Dec 07 16:20:20 EST 2010},
> sc.socialContactStatus=sc.socialContactStatus(1.0)={unread},
> sc.socialContactStatusUserId=sc.socialContactStatusUserId(1.0)={}, sc.soc
> ialContactStatusDate=sc.socialContactStatusDate(1.0)={Tue Dec 07 16:20:20
> EST 2010}, sc.tags=sc.tags(1.0)={[]}, sc.authorId=sc.authorId(1.0)={},
> sc.replyToId=sc.replyTo
> Id(1.0)={}, sc.replyToAuthor=sc.replyToAuthor(1.0)={},
> sc.replyToAuthorId=sc.replyToAuthorId(1.0)={},
> sc.feedId=sc.feedId(1.0)={[124852]}, filterResult_124932_ti=filter
> Result_124932_ti(1.0)={67},
> filterStatus_124932_s=filterStatus_124932_s(1.0)={COMPLETED},
> filterResult_124937_ti=filterResult_124937_ti(1.0)={67},
> filterStatus_124937_s
> =filterStatus_124937_s(1.0)={COMPLETED},
> campaignDateAdded_124957_tdt=campaignDateAdded_124957_tdt(1.0)={Tue Dec 07
> 16:20:20 EST 2010}, campaignStatus_124957_s=campaign
> Status_124957_s(1.0)={NEW},
> campaignDateAdded_124947_tdt=campaignDateAdded_124947_tdt(1.0)={Tue Dec 07
> 16:20:20 EST 2010}, campaignStatus_124947_s=campaignStatus_124947
> _s(1.0)={NEW},
> sc.campaignResultsSummary=sc.campaignResultsSummary(1.0)={[NEW, NEW]}}]
> at
> org.apache.solr.handler.BinaryUpdateRequestHandler$2.document(BinaryUpdateRequestHandler.java:81)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:136)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readIterator(JavaBinUpdateRequestCodec.java:126)
> at
> org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:210)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readNamedList(JavaBinUpdateRequestCodec.java:112)
> at
> org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:175)
> at
> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:101)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:141)
> at
> org.apache.solr.handler.BinaryUpdateRequestHandler.parseAndLoadDocs(BinaryUpdateRequestHandler.java:68)
> at
> org.apache.solr.handler.BinaryUpdateRequestHandler.access$000(BinaryUpdateRequestHandler.java:46)
> at
> org.apache.solr.handler.BinaryUpdateRequestHandler$1.load(BinaryUpdateRequestHandler.java:55)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:723)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>
