Hi there,
I'm looking at some problems we are having with some legacy code which uses
Solr (1.3) under the hood. We seem to get repeated OutOfMemory errors on a
24-48 hour basis searching a relatively small index of 70,000 documents. This
may be an error in the way Solr is configured, or it may be a problem with the
way it's being invoked, I don't know, as I'm fairly new to Solr.
The OutOfMemoryError is an unusual one:
java.lang.OutOfMemoryError: GC overhead limit exceeded
There are also a few stack traces in the solr logs. The first is:
SEVERE: Error during auto-warming of
key:root_show_stop_range:[2009-12-22T08:59:37.254 TO
*]:java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:215)
at
org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122)
at
org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:167)
at
org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:251)
at
org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:218)
at
org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:55)
at
org.apache.lucene.index.MultiSegmentReader$MultiTermDocs.termDocs(MultiSegmentReader.java:609)
at
org.apache.lucene.index.MultiSegmentReader$MultiTermDocs.next(MultiSegmentReader.java:560)
at
org.apache.lucene.search.RangeFilter.getDocIdSet(RangeFilter.java:268)
at
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.<init>(ConstantScoreQuery.java:116)
at
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:81)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:131)
at org.apache.lucene.search.Searcher.search(Searcher.java:126)
at
org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:601)
at
org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:507)
at
org.apache.solr.search.SolrIndexSearcher.cacheDocSet(SolrIndexSearcher.java:482)
at
org.apache.solr.search.SolrIndexSearcher$1.regenerateItem(SolrIndexSearcher.java:224)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:194)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1518)
at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1018)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
I took a memory dump of the running application, and found that vast amounts of
memory (over 400MB) was being consumed by nine or ten large SolrIndexSearcher
objects; references to which are being held within a LinkedHashMap in SolrCore
called "infoRegistry".
I had a quick look at the Solr 1.3.0 source code to try and figure out what was
going wrong and whether SolrCore was being used incorrectly in our own source.
It looks like whenever a new Searcher is created, it "registers" itself with
SolrCore, and this registration places a reference to the Searcher in a
LinkedHashMap (the "infoRegistry").
What is puzzling me is why so many SolrIndexSearcher objects are being created,
and what the conditions are for their creation and removal. The code I can see
in our own product does not use SolrIndexSearcher directly, it simply makes
calls to "execute" on SolrCore; so I would normally expect that that class
would be managing the Searcher life cycle internally.
Does anyone have any idea as to what may be going on here? I have had a look at
the solrconfig.xml file, but it does not seem to depart significantly from the
defaults provided.
Thanks in advance for any help.
Jon
