Hi; According to Sun, the error happens "if too much time is being spent in garbage collection: if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered, an OutOfMemoryError will be thrown.". Specifying more memory should be helpful. On the other hand you should check here: http://wiki.apache.org/solr/SolrPerformanceProblems and here: http://wiki.apache.org/solr/ShawnHeisey
Thanks; Furkan KAMACI 2014-04-09 4:25 GMT+03:00 François Schiettecatte <[email protected]>: > Have you tried using: > > -XX:-UseGCOverheadLimit > > François > > On Apr 8, 2014, at 6:06 PM, Haiying Wang <[email protected]> wrote: > > > Hi, > > > > We were trying to merge a large index (9GB, 21 million docs) into > current index (only 13MB), using mergeindexes command ofCoreAdminHandler, > but always run into OOM error. We currently set the max heap size to 4GB > for the Solr server. We are using 4.6.0, and did not change the original > solrconfig.xml. > > > > Is there any setting/configure that could help to complete the > mergeindexes process without running into OOM error? I can increase the max > jvm heap size, but am afraid that may not scale in case larger index need > to be merged in the future, and hoping the index merge can be performed > with limited memory foorprint. Please help. Thanks! > > > > The jvm heap setting: -Xmx4096M -Xms512M > > > > Command used: > > > > > > curl " > http://dev101:8983/solr/admin/cores?action=mergeindexes&core=collection1&indexDir=/solr/tmp/data/snapshot.20140407194442777 > " > > > > OOM error stack trace: > > > > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded > > at > > java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133) > > at java.lang.StringCoding.decode(StringCoding.java:179) > > at java.lang.String.<init>(String.java:483) > > at java.lang.String.<init>(String.java:539) > > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.readField(CompressingStoredFieldsReader.java:187) > > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:351) > > at > org.apache.lucene.index.SegmentReader.document(SegmentReader.java:276) > > at > > org.apache.lucene.index.IndexReader.document(IndexReader.java:436) > > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:345) > > at > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:316) > > at > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:94) > > at > org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2555) > > at > org.apache.solr.update.DirectUpdateHandler2.mergeIndexes(DirectUpdateHandler2.java:449) > > at > org.apache.solr.update.processor.RunUpdateProcessor.processMergeIndexes(RunUpdateProcessorFactory.java:88) > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processMergeIndexes(UpdateRequestProcessor.java:59) > > at > org.apache.solr.update.processor.LogUpdateProcessor.processMergeIndexes(LogUpdateProcessorFactory.java:149) > > at > org.apache.solr.handler.admin.CoreAdminHandler.handleMergeAction(CoreAdminHandler.java:384) > > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:188) > > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) > > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) > > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) > > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) > > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) > > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) > > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) > > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > > > > Regards, > > > > Haiying > >
