Why stop at 1G? But no, it's really all-or-nothing when you blast a file at Solr. But be sure you're bumping the _solr_ heap, not just Tomcat's heap.
Best Erick On Wed, Mar 21, 2012 at 5:42 PM, vybe3142 <vybe3...@gmail.com> wrote: > While waiting for someohe to help answer my multicore config issue :),... I > decided to test SOLR's limits on a single instance/core config. > > We occasionally need to index large text files (that must not be broken up). > This results in an out of memory error. I tried increasing tomcat's heap > size to 1 GB but this doesn't help. Is there an alternative approach that > addresses this issue? > > The following trace was caused by an attempt to index a 300 MB text file. > > Mar 21, 2012 4:38:33 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2882) > at > java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) > at > java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515) > at java.lang.StringBuilder.append(StringBuilder.java:189) > at > org.apache.solr.handler.extraction.SolrContentHandler.characters(SolrContentHandler.java:292) > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) > at > org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:222) > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) > at > org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46) > at > org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82) > at > org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140) > at > org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:294) > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:268) > at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:132) > at > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242) > at > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242) > at > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:129) > at > org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:213) > at > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > at > org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1372) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Tika-crashing-when-attempting-to-index-large-files-tp3846939p3846939.html > Sent from the Solr - User mailing list archive at Nabble.com.
