Still, 1024M ought to be enough to load one field's norms (how many fields have norms?). If you do things requiring FieldCache that'll also consume RAM.
It's also possible you're hitting this bug (false OOME) in Sun's JRE: http://issues.apache.org/jira/browse/LUCENE-1566 Feel free to go vote for it! Mike On Fri, Mar 27, 2009 at 10:11 PM, Otis Gospodnetic <otis_gospodne...@yahoo.com> wrote: > > That's a tiny heap. Part of it is used for indexing, too. And the fact that > your heap is so small shows you are not really making use of that nice > ramBufferSizeMB setting. :) > > Also, use omitNorms="true" for fields that don't need norms (if their types > don't already do that). > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- >> From: vivek sar <vivex...@gmail.com> >> To: solr-user@lucene.apache.org >> Sent: Friday, March 27, 2009 6:15:59 PM >> Subject: OOM at MultiSegmentReader.norms >> >> Hi, >> >> I've index of size 50G (around 100 million documents) and growing - >> around 2000 records (1 rec = 500 byes) are being written every second >> continuously. If I make any search on this index I get OOM. I'm using >> default cache settings (512,512,256) in the solrconfig.xml. The search >> is using the admin interface (returning 10 rows) with no sorting, >> faceting or highlighting. Max heap size is 1024m. >> >> Mar 27, 2009 9:13:41 PM org.apache.solr.common.SolrException log >> SEVERE: java.lang.OutOfMemoryError: Java heap space >> at >> org.apache.lucene.index.MultiSegmentReader.norms(MultiSegmentReader.java:335) >> at >> org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:69) >> at >> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132) >> at org.apache.lucene.search.Searcher.search(Searcher.java:126) >> at org.apache.lucene.search.Searcher.search(Searcher.java:105) >> at >> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966) >> at >> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838) >> at >> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269) >> at >> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160) >> at >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169) >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) >> at >> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> >> What could be the problem? >> >> Thanks, >> -vivek > >
