Hi,
sounds like you hit a Full GC. Check your GC.log.
Ugo
On 17 Apr 2015 08:24, "Modassar Ather" <modather1...@gmail.com> wrote:
> Hi,
>
> Any suggestion will be really helpful. Kindly provide your inputs.
>
> Thanks,
> Modassar
>
> On Thu, Apr 16, 2015 at 4:27 PM, Modassar Ather <modather1...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I have a setup of 5 node SolrCloud (Lucene/Solr version 5.1.0) without
> > replicas. When I am executing complex and large queries with wild-cards
> > after some time I am getting following exceptions.
> > The index size on each of the node is around 170GB and the memory is set
> > to -Xms20g -Xmx24g on each node.
> >
> > Empty shard!
> > org.apache.solr.common.SolrException log
> > SEVERE: org.apache.solr.common.SolrException: no servers hosting shard:
> > at
> >
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:214)
> > at
> >
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:184)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > There is no OutofMemory or any other major lead for me to understand what
> > had caused it. May be I am missing something. There are following other
> > exceptions:
> >
> > SEVERE: null:org.apache.solr.common.SolrException:
> > org.apache.solr.client.solrj.SolrServerException: Timeout occurred while
> > waiting response from server at: http://<server>:8080/solr/collection
> > at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:342)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> > at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> > at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> > at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> > at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
> > at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> > at
> > org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
> > at
> >
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
> > at
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> > WARNING: listener throws error
> > org.apache.solr.common.SolrException:
> > org.apache.zookeeper.KeeperException$SessionExpiredException:
> > KeeperErrorCode = Session expired for /configs/collection/params.json
> > at
> >
> org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:163)
> > at
> > org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:919)
> > at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2500)
> > at org.apache.solr.cloud.ZkController$4.run(ZkController.java:2366)
> > Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException:
> > KeeperErrorCode = Session expired for /configs/collection/params.json
> > at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> > at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> > at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
> > at
> >
> org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:294)
> > at
> >
> org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:291)
> > at
> >
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> > at
> > org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)
> > at
> >
> org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:153)
> > ... 3 more
> >
> > The Zookeeper session timeout is set to 30000. In the log file I can see
> > logs of the following pattern for all the queries I fired.
> > INFO: [collection] webapp=/solr path=/search_handler
> > params={sort=score+desc&start=0&q=(ft:search term)} status=0 QTime=<time>
> > If I am not wrong they are getting executed but somehow as the shard is
> > gone down which I can see in /clusterstate.json under the log, the search
> > is failing.
> >
> > Kindly help me debug/solve this problem.
> >
> > Regards,
> > Modassar
> >
>
