Yes - more RAM is not a solution to your problem. Jonathan Ariel wrote: > You are saying that I should give more memory than 12GB? > When I was with 10GB I had the exceptions that I sent. Switching to 12GB > made them disappear. > So I think I don't have problems with FieldCache any more. What it seems > like a problem is 11% on the application time dedicated to GC. Specially > when those servers are under really heavy load. > I think that's why I sometimes get queries that in one moment are being > executed in a few ms and a moment after 20 seconds! > > It seems like I should tune my jvm, don't you think so? > > On Fri, Sep 25, 2009 at 1:01 PM, Fuad Efendi <f...@efendi.ca> wrote: > > >> Give it even more memory. >> >> Lucene FieldCache is used to store non-tokenized single-value non-boolean >> (DocumentId -> FieldValue) pairs, and it is used (in-full!) for instance >> for >> sorting query results. >> >> So that if you have 100,000,000 documents with specific heavily distributed >> field values (cardinality is high! Size is 100bytes!) you need >> 10,000,000,000 bytes for just this instance of FieldCache. >> >> GC does not play any role. FieldCache won't be GC-collected. >> >> >> -Fuad >> http://www.linkedin.com/in/liferay >> >> >> >> >>> -----Original Message----- >>> From: Jonathan Ariel [mailto:ionat...@gmail.com] >>> Sent: September-25-09 11:37 AM >>> To: solr-user@lucene.apache.org; yo...@lucidimagination.com >>> Subject: Re: Solr and Garbage Collection >>> >>> Right, now I'm giving it 12GB of heap memory. >>> If I give it less (10GB) it throws the following exception: >>> >>> Sep 5, 2009 7:18:32 PM org.apache.solr.common.SolrException log >>> SEVERE: java.lang.OutOfMemoryError: Java heap space >>> at >>> >>> >> org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:3 >> 61 >> >>> ) >>> at >>> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72) >>> at >>> >>> >> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:3 >> 52 >> >>> ) >>> at >>> >>> >> org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:2 >> 67 >> >>> ) >>> at >>> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:185) >>> at >>> >>> >> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:2 >> 07 >> >>> ) >>> at >>> >>> >> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:104) >> >>> at >>> >>> >> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java >> :7 >> >>> 0) >>> at >>> >>> >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand >> le >> >>> r.java:169) >>> at >>> >>> >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. >> ja >> >>> va:131) >>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) >>> at >>> >>> >> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3 >> 03 >> >>> ) >>> at >>> >>> >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: >> 23 >> >>> 2) >>> at >>> >>> >> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler >> .j >> >>> ava:1089) >>> at >>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) >>> at >>> >>> >> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) >> >>> at >>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) >>> at >>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) >>> at >>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) >>> at >>> >>> >> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerColl >> ec >> >>> tion.java:211) >>> at >>> >>> >> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:11 >> 4) >> >>> at >>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) >>> at org.mortbay.jetty.Server.handle(Server.java:285) >>> at >>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) >>> at >>> >>> >> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java: >> 83 >> >>> 5) >>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) >>> at >>> >> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) >> >>> at >>> >> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) >> >>> at >>> >>> >> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:22 >> 6) >> >>> at >>> >>> >> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:4 >> 42 >> >>> ) >>> >>> On Fri, Sep 25, 2009 at 10:55 AM, Yonik Seeley >>> <yo...@lucidimagination.com>wrote: >>> >>> >>>> On Fri, Sep 25, 2009 at 9:30 AM, Jonathan Ariel <ionat...@gmail.com> >>>> wrote: >>>> >>>>> Hi to all! >>>>> Lately my solr servers seem to stop responding once in a while. I'm >>>>> >> using >> >>>>> solr 1.3. >>>>> Of course I'm having more traffic on the servers. >>>>> So I logged the Garbage Collection activity to check if it's because >>>>> >> of >> >>>>> that. It seems like 11% of the time the application runs, it is >>>>> >> stopped >> >>>>> because of GC. And some times the GC takes up to 10 seconds! >>>>> Is is normal? My instances run on a 16GB RAM, Dual Quad Core Intel >>>>> >> Xeon >> >>>>> servers. My index is around 10GB and I'm giving to the instances 10GB >>>>> >> of >> >>>>> RAM. >>>>> >>>> Bigger heaps lead to bigger GC pauses in general. >>>> Do you mean that you are giving the JVM a 10GB heap? Were you getting >>>> OOM exceptions with a smaller heap? >>>> >>>> -Yonik >>>> http://www.lucidimagination.com >>>> >>>> >> >> > >
-- - Mark http://www.lucidimagination.com
