Give it even more memory. Lucene FieldCache is used to store non-tokenized single-value non-boolean (DocumentId -> FieldValue) pairs, and it is used (in-full!) for instance for sorting query results.
So that if you have 100,000,000 documents with specific heavily distributed field values (cardinality is high! Size is 100bytes!) you need 10,000,000,000 bytes for just this instance of FieldCache. GC does not play any role. FieldCache won't be GC-collected. -Fuad http://www.linkedin.com/in/liferay > -----Original Message----- > From: Jonathan Ariel [mailto:ionat...@gmail.com] > Sent: September-25-09 11:37 AM > To: solr-user@lucene.apache.org; yo...@lucidimagination.com > Subject: Re: Solr and Garbage Collection > > Right, now I'm giving it 12GB of heap memory. > If I give it less (10GB) it throws the following exception: > > Sep 5, 2009 7:18:32 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.OutOfMemoryError: Java heap space > at > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:3 61 > ) > at > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72) > at > org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:3 52 > ) > at > org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:2 67 > ) > at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:185) > at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:2 07 > ) > at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:104) > at > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java :7 > 0) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand le > r.java:169) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. ja > va:131) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3 03 > ) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: 23 > 2) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .j > ava:1089) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > at > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerColl ec > tion.java:211) > at > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:11 4) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > at org.mortbay.jetty.Server.handle(Server.java:285) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) > at > org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java: 83 > 5) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) > at > org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:22 6) > at > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:4 42 > ) > > On Fri, Sep 25, 2009 at 10:55 AM, Yonik Seeley > <yo...@lucidimagination.com>wrote: > > > On Fri, Sep 25, 2009 at 9:30 AM, Jonathan Ariel <ionat...@gmail.com> > > wrote: > > > Hi to all! > > > Lately my solr servers seem to stop responding once in a while. I'm using > > > solr 1.3. > > > Of course I'm having more traffic on the servers. > > > So I logged the Garbage Collection activity to check if it's because of > > > that. It seems like 11% of the time the application runs, it is stopped > > > because of GC. And some times the GC takes up to 10 seconds! > > > Is is normal? My instances run on a 16GB RAM, Dual Quad Core Intel Xeon > > > servers. My index is around 10GB and I'm giving to the instances 10GB of > > > RAM. > > > > Bigger heaps lead to bigger GC pauses in general. > > Do you mean that you are giving the JVM a 10GB heap? Were you getting > > OOM exceptions with a smaller heap? > > > > -Yonik > > http://www.lucidimagination.com > >
