Are there any distrib facet gurus on the list? I would be ready to try sensible ideas, including on the source code level, if someone of you could give me a hand.
Dmitry On Wed, Apr 24, 2013 at 3:08 PM, Dmitry Kan <solrexp...@gmail.com> wrote: > Hello list, > > We deal with an anomaly when doing a distributed facet query against 102 > shards. > > The problem manifests itself in both the frontend solr (router) and a > shard. Each time the request is executed, always different shard is > affected (at random, hence the "anomaly"). > > The query is: http://router_host:router_port > /solr/select?q=test&facet=true&facet.field=field_of_type_long&facet.limit=1330&facet.mincount=1&rows=1&facet.sort=index&facet.zeros=false&facet.offset=0 > I have omitted the shards parameter. > > The router log: > > request: http://10.155.244.181:9150/solr/select > at > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:430) > at > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244) > at > org.apache.solr.handler.component.HttpCommComponent$1.call(SearchHandler.java:421) > at > org.apache.solr.handler.component.HttpCommComponent$1.call(SearchHandler.java:393) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > > Notice the port of a shard, that is affected. That port changes all the > time, even for the same request > The log entry is prepended with lines: > > SEVERE: org.apache.solr.common.SolrException: Internal Server Error > > Internal Server Error > > (they are not in the pastebin link) > > The shard log: > > Apr 24, 2013 11:08:49 AM org.apache.solr.common.SolrException log > SEVERE: java.lang.NullPointerException > at java.io.StringReader.<init>(StringReader.java:50) > at > org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:203) > at > org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:80) > at org.apache.solr.search.QParser.getQuery(QParser.java:142) > at > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:81) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454) > at java.lang.Thread.run(Thread.java:722) > > Apr 24, 2013 11:08:49 AM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/select params={} status=500 QTime=2 > Apr 24, 2013 11:08:49 AM org.apache.solr.common.SolrException log > SEVERE: java.lang.NullPointerException > at java.io.StringReader.<init>(StringReader.java:50) > at > org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:203) > at > org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:80) > at org.apache.solr.search.QParser.getQuery(QParser.java:142) > at > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:81) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454) > at java.lang.Thread.run(Thread.java:722) > > > Essentially, for some reason, the shard has received an empty request. The > hypothesis we have is: the router sends two queries, of which second is > empty. The first one being the original request, the second being the > request with doc ids. Is this correct? > > GET limitations should not be affecting here, as the router sends a > distributed query via POST. > > In the attempt to solve the issue we have tried modifying the shard > timeout settings on the router. Currently set values are: > shard-connection-timeout=300000&shard-socket-timeout=300000 > > This didn't help though. > > What could be tried else? > > Thanks, > > Dmitry > >
