I have been facing this issue since long in production environment and wanted
to know if anybody came across can share their thoughts.
Appreciate your help.
Environment
2 GB index file
3.5 million documents
15 mins. time interval for committing 100 to 400 document updates
Commit happens once in 15 mins.
3.5 GB of RAM available for JVM
Solr Version 1.3 ; (nightly build of oct 18, 2008)
MDB - Message Driven Bean
I am Not using solr's replication mecahnism. Also don't use xml post update
since the amount of data is too much.
I have bundled a MDB that receives messages for data updates and uses solr's
update handler to update and commit the index.
Optimize happens once a day.
Everything runs fine for 2-3 days; after that I keep getting following
exceptions.
Exception
org.apache.solr.common.SolrException log java.lang.OutOfMemoryError:
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:350)
at
org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:596)
at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:136)
at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:92)
at org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:907)
at
org.apache.lucene.index.MultiSegmentReader.norms(MultiSegmentReader.java:338)
at
org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:69)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:131)
at org.apache.lucene.search.Searcher.search(Searcher.java:126)
at org.apache.lucene.search.Searcher.search(Searcher.java:105)
at
org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1170)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:856)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:283)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:170)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1302)
at
org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:51)
at org.apache.solr.core.SolrCore$4.call(SolrCore.java:1128)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
at java.lang.Thread.run(Thread.java:810)
