Thanks Otis and Mike. I'm indexing total of 9 fields, with 5 having norms turned on. I think I may not need it and will try use the omitNorms for them.
How do I make use of RAMBuffer in Solr? I couldn't find anything on this on the Wiki - any pointer? Thanks, -vivek On Sat, Mar 28, 2009 at 1:09 AM, Michael McCandless <luc...@mikemccandless.com> wrote: > Still, 1024M ought to be enough to load one field's norms (how many > fields have norms?). If you do things requiring FieldCache that'll > also consume RAM. > > It's also possible you're hitting this bug (false OOME) in Sun's JRE: > > http://issues.apache.org/jira/browse/LUCENE-1566 > > Feel free to go vote for it! > > Mike > > On Fri, Mar 27, 2009 at 10:11 PM, Otis Gospodnetic > <otis_gospodne...@yahoo.com> wrote: >> >> That's a tiny heap. Part of it is used for indexing, too. And the fact >> that your heap is so small shows you are not really making use of that nice >> ramBufferSizeMB setting. :) >> >> Also, use omitNorms="true" for fields that don't need norms (if their types >> don't already do that). >> >> Otis >> -- >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> >> >> >> ----- Original Message ---- >>> From: vivek sar <vivex...@gmail.com> >>> To: solr-user@lucene.apache.org >>> Sent: Friday, March 27, 2009 6:15:59 PM >>> Subject: OOM at MultiSegmentReader.norms >>> >>> Hi, >>> >>> I've index of size 50G (around 100 million documents) and growing - >>> around 2000 records (1 rec = 500 byes) are being written every second >>> continuously. If I make any search on this index I get OOM. I'm using >>> default cache settings (512,512,256) in the solrconfig.xml. The search >>> is using the admin interface (returning 10 rows) with no sorting, >>> faceting or highlighting. Max heap size is 1024m. >>> >>> Mar 27, 2009 9:13:41 PM org.apache.solr.common.SolrException log >>> SEVERE: java.lang.OutOfMemoryError: Java heap space >>> at >>> org.apache.lucene.index.MultiSegmentReader.norms(MultiSegmentReader.java:335) >>> at >>> org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:69) >>> at >>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132) >>> at org.apache.lucene.search.Searcher.search(Searcher.java:126) >>> at org.apache.lucene.search.Searcher.search(Searcher.java:105) >>> at >>> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966) >>> at >>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838) >>> at >>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269) >>> at >>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160) >>> at >>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169) >>> at >>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) >>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) >>> at >>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) >>> at >>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>> at >>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >>> at >>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>> at >>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) >>> at >>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >>> at >>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >>> >>> What could be the problem? >>> >>> Thanks, >>> -vivek >> >> >
