Have you tried using:
-XX:-UseGCOverheadLimit François On Apr 8, 2014, at 6:06 PM, Haiying Wang <haiyingwa...@yahoo.com> wrote: > Hi, > > We were trying to merge a large index (9GB, 21 million docs) into current > index (only 13MB), using mergeindexes command ofCoreAdminHandler, but always > run into OOM error. We currently set the max heap size to 4GB for the Solr > server. We are using 4.6.0, and did not change the original solrconfig.xml. > > Is there any setting/configure that could help to complete the mergeindexes > process without running into OOM error? I can increase the max jvm heap size, > but am afraid that may not scale in case larger index need to be merged in > the future, and hoping the index merge can be performed with limited memory > foorprint. Please help. Thanks! > > The jvm heap setting: -Xmx4096M -Xms512M > > Command used: > > > curl > "http://dev101:8983/solr/admin/cores?action=mergeindexes&core=collection1&indexDir=/solr/tmp/data/snapshot.20140407194442777"; > > OOM error stack trace: > > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded > at > java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133) > at java.lang.StringCoding.decode(StringCoding.java:179) > at java.lang.String.<init>(String.java:483) > at java.lang.String.<init>(String.java:539) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.readField(CompressingStoredFieldsReader.java:187) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:351) > at > org.apache.lucene.index.SegmentReader.document(SegmentReader.java:276) > at > org.apache.lucene.index.IndexReader.document(IndexReader.java:436) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:345) > at > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:316) > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:94) > at > org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2555) > at > org.apache.solr.update.DirectUpdateHandler2.mergeIndexes(DirectUpdateHandler2.java:449) > at > org.apache.solr.update.processor.RunUpdateProcessor.processMergeIndexes(RunUpdateProcessorFactory.java:88) > at > org.apache.solr.update.processor.UpdateRequestProcessor.processMergeIndexes(UpdateRequestProcessor.java:59) > at > org.apache.solr.update.processor.LogUpdateProcessor.processMergeIndexes(LogUpdateProcessorFactory.java:149) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleMergeAction(CoreAdminHandler.java:384) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:188) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > > Regards, > > Haiying
signature.asc
Description: Message signed with OpenPGP using GPGMail
