org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:403)
- this piece of code do not request Array[100M] (as I seen with
Lucene), it asks only few bytes / Kb for a field...
Probably 128 - 512 is not enough; it is also advisable to use equal sizes
-Xms1024M -Xmx1024M
(it minimizes GC frequency, and itensures that 1024M is available at startup)
OOM happens also with fragmented memory, when application requests big
contigues fragment and GC is unable to optimize; looks like your
application requests a little and memory is not available...
Quoting sundar shankar <[EMAIL PROTECTED]>:
From: [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Subject: Out of memory on Solr sorting
Date: Tue, 22 Jul 2008 19:11:02 +0000
Hi,
Sorry again fellos. I am not sure whats happening. The day with
solr is bad for me I guess. EZMLM didnt let me send any mails this
morning. Asked me to confirm subscription and when I did, it said I
was already a member. Now my mails are all coming out bad. Sorry
for troubling y'all this bad. I hope this mail comes out right.
Hi,
We are developing a product in a agile manner and the current
implementation has a data of size just about a 800 megs in dev.
The memory allocated to solr on dev (Dual core Linux box) is 128-512.
My config
=========
<!-- autocommit pending docs if certain criteria are met
<autoCommit>
<maxDocs>10000</maxDocs>
<maxTime>1000</maxTime>
</autoCommit>
-->
<filterCache
class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="256"/>
<queryResultCache
class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="256"/>
<documentCache
class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
<enableLazyFieldLoading>true</enableLazyFieldLoading>
My Field
=======
<fieldType name="autocomplete" class="solr.TextField">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="([^a-z0-9])" replacement="" replace="all" />
<filter class="solr.EdgeNGramFilterFactory"
maxGramSize="100" minGramSize="1" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="([^a-z0-9])" replacement="" replace="all" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="^(.{20})(.*)?" replacement="$1" replace="all" />
</analyzer>
</fieldType>
Problem
======
I execute a query that returns 24 rows of result. I pick 10 out of
it. I have no problem when I execute this.
But When I do sort it by a String field that is fetched from this
result. I get an OOM. I am able to execute several
other queries with no problem. Just having a sort asc clause added
to the query throws an OOM. Why is that.
What should I have ideally done. My config on QA is pretty similar
to the dev box and probably has more data than on dev.
It didnt throw any OOM during the integration test. The Autocomplete
is a new field we added recently.
Another point is that the indexing is done with a field of type string
<field name="XXX" type="string" indexed="true" stored="true"
termVectors="true"/>
and the autocomplete field is a copy field.
The sorting is done based on string field.
Please do lemme know what mistake am I doing?
Regards
Sundar
P.S: The stack trace of the exception is
Caused by: org.apache.solr.client.solrj.SolrServerException: Error
executing query
at
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:86)
at
org.apache.solr.client.solrj.impl.BaseSolrServer.query(BaseSolrServer.java:101)
at
com.apollo.sisaw.solr.service.AbstractSolrSearchService.makeSolrQuery(AbstractSolrSearchService.java:193)
... 105 more
Caused by: org.apache.solr.common.SolrException: Java heap space
java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:403)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:352)
at
org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSortedHitQueue.java:416)
at
org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:207)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)
at
org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:156)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:128)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1025)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
_________________________________________________________________
Wish to Marry Now? Click Here to Register FREE
http://www.shaadi.com/registration/user/index.php?ptnr=mhottag
