Can you isolate any exceptions that happened just before that exception.
started repeating?
- Mark
> On Nov 7, 2013, at 9:09 AM, Eric Bus <eric....@websight.nl> wrote:
>
> Hi,
>
> I'm having a problem with one of my shards. Since yesterday, SOLR keeps
> repeating the same exception over and over for this shard.
> The webinterface for this SOLR instance is also not working (it hangs on the
> Loading indicator).
>
> Nov 7, 2013 9:08:12 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: [website1_shard1_replica3] webapp=/solr path=/update
> params={update.distrib=TOLEADER&wt=javabin&version=2} {} 0 0
> Nov 7, 2013 9:08:12 AM org.apache.solr.common.SolrException log
> SEVERE: java.lang.RuntimeException: SolrCoreState already closed
> at
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:79)
> at
> org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandler2.java:276)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:77)
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:55)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:460)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionDelete(DistributedUpdateProcessor.java:1036)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:721)
> at
> org.apache.solr.update.processor.LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:121)
> at
> org.apache.solr.handler.loader.XMLLoader.processDelete(XMLLoader.java:346)
> at
> org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:277)
> at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
> at
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:448)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:269)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> at java.lang.Thread.run(Thread.java:662)
>
> I have about 3GB of logfiles for this single message. Reloading the
> collection does not work. Reloading the specific shard core returns the same
> exception. The only option seems to be to restart the server. But because
> it's the leader for a lot of collections, I want to know why this is
> happening. I've seen this problem before, and I haven't figured out what is
> causing it.
>
> I've reported a different problem a few days ago with 'hanging' deleted
> logfiles. Could this be related? Could the hanging logfiles prevent a new
> Searcher from opening? I've updated two of my three hosts to 4.5.1 but after
> only 2 days uptime, I'm still seeing about 11.000 deleted logfiles in the
> lsof output.
>
> Best regards,
> Eric Bus
>
>
