There's a Lucene/Solr memory size estimator spreadsheet in the SVN: http://svn.apache.org/repos/asf/lucene/dev/trunk/dev-tools/size-estimator-lucene-solr.xls
Dan On Mon, Apr 30, 2012 at 11:39 AM, Yuval Dotan <yuvaldo...@gmail.com> wrote: > Thanks for the fast answer > One more question: > Is there a way to know (some formula) what is the size of memory i need for > these actions? > > Thanks > Yuval > > On Mon, Apr 30, 2012 at 11:50, Dan Tuffery <dan.tuff...@gmail.com> wrote: > > > You need to add more memory to the JVM that is running Solr: > > > > http://wiki.apache.org/solr/SolrPerformanceFactors#OutOfMemoryErrors > > > > Dan > > > > On Mon, Apr 30, 2012 at 9:43 AM, Yuval Dotan <yuvaldo...@gmail.com> > wrote: > > > > > Hi Guys > > > I have a problem and i need your assistance > > > I get an exception when doing field cache faceting (the enum method > works > > > perfectly): > > > > > > */solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10* > > > > > > <lst name="error"> > > > <str name="msg">java.lang.OutOfMemoryError: Java heap space</str> > > > <str name="trace"> > > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277) > > > at > > > > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) > > > at > > > > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) > > > at > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > > > at > > > > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) > > > at > > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) > > > at > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) > > > at > > > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) > > > at > > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > > > at > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) > > > at > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > > > at > > > > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) > > > at > > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) > > > at > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) > > > at org.eclipse.jetty.server.Server.handle(Server.java:351) at > > > > > > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) > > > at > > > > > > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) > > > at > > > > > > org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890) > > > at > > > > > > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944) > > > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at > > > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) > at > > > > > > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) > > > at > > > > > > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) > > > at > > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) > > > at > > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) > > > at java.lang.Thread.run(Thread.java:679) Caused by: > > > java.lang.OutOfMemoryError: Java heap space at > > > org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at > > > > org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267) > > at > > > > org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81) > > at > > > > > > org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178) > > > at > > > > > > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248) > > > at > > > > > > org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081) > > > at > > > > > > org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077) > > > at > > > > > > org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459) > > > at > > > > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310) > > > at > > > > > > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) > > > at > > > > > > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) > > > at > > > > > > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) > > > at > > > > > > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) > > > at > > > > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256) > > > at > > > > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) > > > at > > > > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) > > > at > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > > > at > > > > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) > > > at > > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) > > > at > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) > > > at > > > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) > > > at > > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > > > at > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) > > > at > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > > > at > > > > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) > > > at > > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) > > > at > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) > > > at org.eclipse.jetty.server.Server.handle(Server.java:351) at > > > > > > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) > > > </str> > > > <int name="code">500</int> > > > </lst> > > > > > > my index size : 2,083,704,454 docs > > > > > > attached my solrconfig.xml and schema.xml > > > > > > I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT > > > support) with RankingAlgorithm 1.4.2 (war file 33MB) < > > http://solr-ra.tgels.com/solr-ra.jsp> > > > > > > Thanks > > > Yuval > > > > > > > > >
