After importing 3 million records from DB into Solr 7.2.1 on a CentOS 7, OpenJDK 8, I just restarted Solr and the core (mycore) cannot instantiate. data/index is 36GB and tlog folder contains a file named tlog.0000000000000001862 which is about 20MB.
Here is the full log: INFO (main) [ ] o.e.j.s.Server jetty-9.3.20.v20170531 INFO (main) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.1 INFO (main) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port 8983 INFO (main) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /opt/solr INFO (main) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-13T11:09:55.143Z INFO (main) [ ] o.a.s.u.StartupLoggingUtils Property solr.log.muteconsole given. Muting ConsoleAppender named CONSOLE INFO (main) [ ] o.a.s.c.SolrResourceLoader Using system property solr.solr.home: /var/solr/data INFO (main) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /var/solr/data/solr.xml INFO (main) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /var/solr/data INFO (main) [ ] o.a.s.c.CorePropertiesLocator Cores are: [mycore] INFO (coreLoadExecutor-6-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrResourceLoader [null] Added 5 libs to classloader, from paths: [/var/solr/data/mycore/lib] INFO (main) [ ] o.e.j.s.Server Started @4565ms INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrResourceLoader [mycore] Added 55 libs to classloader, from paths: [/opt/solr/contrib/clustering/lib, /opt/solr/contrib/extraction/lib, /opt/solr/contrib/langid/lib, /opt/solr/contrib/velocity/lib, /opt/solr/dist] INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.1 INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.s.IndexSchema [mycore] Schema name=solr721-mycore WARN (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly. INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.s.IndexSchema Loaded schema solr721-mycore/1.6 with uniqueid field id INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.CoreContainer Creating SolrCore 'mycore' using configuration from instancedir /var/solr/data/mycore, trusted=true INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrCore [[mycore] ] Opening new SolrCore at [/var/solr/data/mycore], dataDir=[/var/solr/data/mycore/data/] INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5 INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrCore [mycore] CLOSING SolrCore org.apache.solr.core.SolrCore@735e855b INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.mycore, tag=1935574363 ERROR (coreContainerWorkExecutor-2-thread-1) [ ] o.a.s.c.CoreContainer Error waiting for SolrCore to be created java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [mycore] at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.solr.core.CoreContainer.lambda$load$14(CoreContainer.java:671) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Unable to create core [mycore] at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1053) at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:642) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) ... 5 more Caused by: org.apache.solr.common.SolrException: Error Instantiating Update Handler, solr.DirectUpdateHandler2 failed to instantiate org.apache.solr.update.UpdateHandler at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1008) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:863) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1037) ... 7 more Caused by: org.apache.solr.common.SolrException: Error Instantiating Update Handler, solr.DirectUpdateHandler2 failed to instantiate org.apache.solr.update.UpdateHandler at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:807) at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:855) at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1108) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:978) ... 9 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:793) ... 12 more Caused by: java.lang.StackOverflowError at org.noggit.CharArr.reserve(CharArr.java:116) at org.apache.solr.common.util.ByteUtils.UTF8toUTF16(ByteUtils.java:68) at org.apache.solr.common.util.JavaBinCodec.readStr(JavaBinCodec.java:868) at org.apache.solr.common.util.JavaBinCodec.readStr(JavaBinCodec.java:857) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:266) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readOrderedMap(JavaBinCodec.java:199) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:274) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readArray(JavaBinCodec.java:747) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:272)
