I am sorry about a type mistake 8,000,000,000 -> 800,000,000
2013/5/27 Jam Luo <cooljam2...@gmail.com> > I have the same problem. at 4.1 ,a solr instance could take 8,000,000,000 > doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at > facet query. the facet field was token by space. > > May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log > SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java > heap space > at > org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) > at org.eclipse.jetty.server.Server.handle(Server.java:350) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) > at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) > at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851) > at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) > at > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.lang.OutOfMemoryError: Java heap space > at > org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448) > at > org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:179) > at > org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664) > at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426) > at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517) > at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252) > at > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1825) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) > at org.eclipse.jetty.server.Server.handle(Server.java:350) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) > at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) > at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851) > > > > 2013/5/19 J Mohamed Zahoor <zah...@indix.com> > >> >> aah… was doing a facet on a double field which was having 6 decimal >> places… >> No surprise that the lucene cache got full… >> >> .z/ahoor >> >> On 17-May-2013, at 11:56 PM, J Mohamed Zahoor <zah...@indix.com> wrote: >> >> > Memory increase a lot with queries which have facets… >> > >> > >> > ./Zahoor >> > >> > >> > On 17-May-2013, at 10:00 PM, Shawn Heisey <s...@elyograg.org> wrote: >> > >> >> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote: >> >>> I moved to 4.2.1 from 4.1 recently.. everything was working fine >> until i added few more stats query.. >> >>> Now i am getting this error frequently that solr does not run even >> for 2 minutes continuously. >> >>> All 5GB is getting used instantaneously in few queries... >> >> >> >> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It >> >> wasn't OOM errors, they were just using a lot more heap than they were >> >> before and running into constant full garbage collections. >> >> >> >> There is another message on this list about someone who upgraded from >> >> 3.5 to 4.2 and is having memory troubles. >> >> >> >> The person on IRC made most of their fields unstored and reindexed, >> >> which fixed the problem for them. They only needed a few fields >> stored. >> >> >> >> Because the IRC user was on 4.0, I originally thought it had something >> >> to do with compressed stored fields, but on this thread, they started >> >> with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version, >> >> then they had compressed stored fields before the upgrade. >> >> >> >> The user on IRC was not using termvectors or docvalues, which would be >> >> potential pain points unique to 4.2. >> >> >> >> I'm using 4.2.1 with no trouble in my setup, but I do have a heap >> that's >> >> considerably larger than I need. There are no apparent memory leaks - >> >> it's been running for over a month with updates once a minute. I've >> >> finally switched over from the 3.5.0 index to the new one, so for the >> >> last few days, it has been also taking our full query load. >> >> >> >> What could have changed between 4.1 and 4.2 to cause dramatically >> >> increased memory usage? >> >> >> >> From my /admin/system: >> >> >> >> <date name="startTime">2013-04-05T15:52:55.751Z</date> >> >> >> >> Thanks, >> >> Shawn >> >> >> > >> >> >
