Vivek, No need for any RAMBuffer stuff, just use the appropriate ramBufferSizeMB value in solrconfig.xml. This will only help with indexing, though!
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: vivek sar <vivex...@gmail.com> > To: solr-user@lucene.apache.org > Sent: Monday, March 30, 2009 12:53:21 PM > Subject: Re: OOM at MultiSegmentReader.norms > > Thanks Otis and Mike. > > I'm indexing total of 9 fields, with 5 having norms turned on. I think > I may not need it and will try use the omitNorms for them. > > How do I make use of RAMBuffer in Solr? I couldn't find anything on > this on the Wiki - any pointer? > > Thanks, > -vivek > > On Sat, Mar 28, 2009 at 1:09 AM, Michael McCandless > wrote: > > Still, 1024M ought to be enough to load one field's norms (how many > > fields have norms?). If you do things requiring FieldCache that'll > > also consume RAM. > > > > It's also possible you're hitting this bug (false OOME) in Sun's JRE: > > > > http://issues.apache.org/jira/browse/LUCENE-1566 > > > > Feel free to go vote for it! > > > > Mike > > > > On Fri, Mar 27, 2009 at 10:11 PM, Otis Gospodnetic > > wrote: > >> > >> That's a tiny heap. Part of it is used for indexing, too. And the fact > >> that > your heap is so small shows you are not really making use of that nice > ramBufferSizeMB setting. :) > >> > >> Also, use omitNorms="true" for fields that don't need norms (if their > >> types > don't already do that). > >> > >> Otis > >> -- > >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > >> > >> > >> > >> ----- Original Message ---- > >>> From: vivek sar > >>> To: solr-user@lucene.apache.org > >>> Sent: Friday, March 27, 2009 6:15:59 PM > >>> Subject: OOM at MultiSegmentReader.norms > >>> > >>> Hi, > >>> > >>> I've index of size 50G (around 100 million documents) and growing - > >>> around 2000 records (1 rec = 500 byes) are being written every second > >>> continuously. If I make any search on this index I get OOM. I'm using > >>> default cache settings (512,512,256) in the solrconfig.xml. The search > >>> is using the admin interface (returning 10 rows) with no sorting, > >>> faceting or highlighting. Max heap size is 1024m. > >>> > >>> Mar 27, 2009 9:13:41 PM org.apache.solr.common.SolrException log > >>> SEVERE: java.lang.OutOfMemoryError: Java heap space > >>> at > >>> > org.apache.lucene.index.MultiSegmentReader.norms(MultiSegmentReader.java:335) > >>> at > >>> org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:69) > >>> at > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132) > >>> at org.apache.lucene.search.Searcher.search(Searcher.java:126) > >>> at org.apache.lucene.search.Searcher.search(Searcher.java:105) > >>> at > >>> > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966) > >>> at > >>> > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838) > >>> at > >>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269) > >>> at > >>> > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160) > >>> at > >>> > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169) > >>> at > >>> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > >>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) > >>> at > >>> > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) > >>> at > >>> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) > >>> at > >>> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > >>> at > >>> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > >>> at > >>> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > >>> at > >>> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > >>> at > >>> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > >>> at > >>> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > >>> at > >>> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > >>> > >>> What could be the problem? > >>> > >>> Thanks, > >>> -vivek > >> > >> > >
