Hi Joel, Yes it worked fine after setting *useFilterForSortedQuery *false. Does SOLR-8291 <https://jira.apache.org/jira/browse/SOLR-8291> address this issue? As per SOLR-8291, it happens based on number of segments. But it my case, q=*:* always works irrespective of number of segments. Queries in specific field sometimes work and sometimes do not based on query.
So, the issue here is because of the documents matching for given query instead of number of segments. On Fri, Sep 28, 2018 at 4:37 AM Joel Bernstein <joels...@gmail.com> wrote: > If you have: <useFilterForSortedQuery>true</useFilterForSortedQuery> > > Then you will get errors with the export handler. > > After you set this to false do the errors go away? > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Thu, Sep 27, 2018 at 6:40 PM Gaini Rajeshwar < > raja.rajeshwar2...@gmail.com> wrote: > > > Is it anyway related to the following ? > > > > https://jira.apache.org/jira/browse/SOLR-8291 > > > > I will test this by turning off > > <useFilterForSortedQuery>true</useFilterForSortedQuery> > > to check if it helps. > > > > On Fri, Sep 28, 2018 at 3:58 AM Joel Bernstein <joels...@gmail.com> > wrote: > > > > > Ok, that stack trace shows where the problem. I'll investigate and > report > > > back. > > > > > > Joel Bernstein > > > http://joelsolr.blogspot.com/ > > > > > > > > > On Thu, Sep 27, 2018 at 4:01 PM Gaini Rajeshwar < > > > raja.rajeshwar2...@gmail.com> wrote: > > > > > > > @Joel: On some of the shards, i am seeing the following error in > logs. > > > > > > > > null:java.lang.NullPointerException > > > > at > > > > org.apache.lucene.util.BitSetIterator.<init>(BitSetIterator.java:61) > > > > at > > > > org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:243) > > > > at > > > > > > org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:222) > > > > at > > > > > > > > > > org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter.java:515) > > > > at > > > > > > > > > > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:179) > > > > at > > > > > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:551) > > > > at > > > > > > org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:222) > > > > at > > > > > > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539) > > > > at > > > > > > > > > > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181) > > > > at > > > > > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:551) > > > > at > > > > > > > > > > org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:220) > > > > at > > > > > > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539) > > > > at > > > > org.apache.solr.handler.ExportWriter.write(ExportWriter.java:218) > > > > at org.apache.solr.core.SolrCore$4.write(SolrCore.java:2686) > > > > at > > > > > > > > > > org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49) > > > > at > > > > > > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787) > > > > at > > > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524) > > > > at > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) > > > > at > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) > > > > at > > > > > > > > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) > > > > at > > > > > > > > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > > > at > > > > > > > > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > > > at > > > > > > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > > > at > > > > > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) > > > > at > > > > > > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > > > at > > > > > > > > > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > > > > at > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > > > at org.eclipse.jetty.server.Server.handle(Server.java:531) > > > > at > > > > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) > > > > at > > > > > > > > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) > > > > at org.eclipse.jetty.io > > > > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) > > > > at org.eclipse.jetty.io > > > > .FillInterest.fillable(FillInterest.java:102) > > > > at org.eclipse.jetty.io > > > > .ChannelEndPoint$2.run(ChannelEndPoint.java:118) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) > > > > at > > > > > > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > On Fri, Sep 28, 2018 at 1:20 AM Gaini Rajeshwar < > > > > raja.rajeshwar2...@gmail.com> wrote: > > > > > > > > > I've tried with 7.3 version also and encountering same issue. > > > > > > > > > > On Thu, Sep 27, 2018 at 11:02 PM Joel Bernstein < > joels...@gmail.com> > > > > > wrote: > > > > > > > > > >> That is weird, I've not encountered this behavior. There has been > > some > > > > >> changes in 7.4 to the export handler, and I'm wondering if a bug > was > > > > >> introduced. The stack trace you are posting is coming from the > node > > > > where > > > > >> the expression is being run. Can you check the logs from the > shards > > to > > > > see > > > > >> if there are errors on the shards? > > > > >> > > > > >> > > > > >> Joel Bernstein > > > > >> http://joelsolr.blogspot.com/ > > > > >> > > > > >> > > > > >> On Thu, Sep 27, 2018 at 12:10 PM Gaini Rajeshwar < > > > > >> raja.rajeshwar2...@gmail.com> wrote: > > > > >> > > > > >> > Also, to mention, if i give q="*:*" it is working fine (which is > > > kind > > > > of > > > > >> > weird) > > > > >> > > > > > >> > On Thu, Sep 27, 2018 at 9:35 PM Gaini Rajeshwar < > > > > >> > raja.rajeshwar2...@gmail.com> wrote: > > > > >> > > > > > >> > > Hi All, > > > > >> > > > > > > >> > > I am using solr 7.4 version. I am trying to export data using > > > > >> streaming > > > > >> > > expressions. > > > > >> > > > > > > >> > > Following is the simple curl query that i am using. I am > > indexing > > > my > > > > >> data > > > > >> > > and testing the following query. > > > > >> > > > > > > >> > > curl --data-urlencode 'expr=search(collection1, > > > > >> > >> q="text:solar", > > > > >> > >> fl="id", > > > > >> > >> sort="id asc", > > > > >> > >> qt="/export")' > > > > http://localhost:8983/solr/collection1/stream > > > > >> > > > > > > >> > > > > > > >> > > when i run above query with initial few documents indexed, it > is > > > > >> working > > > > >> > > fine. After indexing few thousand documents (around , i am > > getting > > > > the > > > > >> > > following error for the same query. > > > > >> > > > > > > >> > > EXCEPTION":"java.util.concurrent.ExecutionException: > > > > >> > >> java.io.IOException: --> > > http://localhost:8983/solr/collection1/ > > > : > > > > An > > > > >> > >> exception has occurred on the server, refer to server log for > > > > >> details." > > > > >> > > > > > > >> > > > > > > >> > > I've looked into my solr logs and the following is the error > > info > > > > >> > > > > > > >> > > java.io.IOException: java.util.concurrent.ExecutionException: > > > > >> > >> java.io.IOException: --> > > http://localhost:8983/solr/collection1/ > > > : > > > > An > > > > >> > >> exception has occurred on the server, refer to server log for > > > > >> details. > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:400) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:275) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:397) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:83) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:209) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:325) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:120) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:71) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787) > > > > >> > >> at > > > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > > > >> > >> at org.eclipse.jetty.server.Server.handle(Server.java:531) > > > > >> > >> at > > > > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) > > > > >> > >> at > > > > >> > >> org.eclipse.jetty.io > > > > >> > > > > > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) > > > > >> > >> at org.eclipse.jetty.io > > > > .FillInterest.fillable(FillInterest.java:102) > > > > >> > >> at org.eclipse.jetty.io > > > > >> .ChannelEndPoint$2.run(ChannelEndPoint.java:118) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) > > > > >> > >> at java.lang.Thread.run(Thread.java:748) > > > > >> > >> Caused by: java.util.concurrent.ExecutionException: > > > > >> java.io.IOException: > > > > >> > >> --> http://localhost:8983/solr/collection1/: An exception > has > > > > >> occurred > > > > >> > >> on the server, refer to server log for details. > > > > >> > >> at > java.util.concurrent.FutureTask.report(FutureTask.java:122) > > > > >> > >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:394) > > > > >> > >> ... 49 more > > > > >> > >> Caused by: java.io.IOException: --> > > > > >> > >> http://localhost:8983/solr/collection1/: An exception has > > > occurred > > > > >> on > > > > >> > >> the server, refer to server log for details. > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:225) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:484) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:507) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:494) > > > > >> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > >> > >> ... 1 more > > > > >> > >> Caused by: java.io.IOException: JSONTupleStream: expected > > > > >> OBJECT_START > > > > >> > >> but got EOF > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.JSONTupleStream.expect(JSONTupleStream.java:99) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.JSONTupleStream.advanceToDocs(JSONTupleStream.java:179) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.JSONTupleStream.next(JSONTupleStream.java:77) > > > > >> > >> at > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:194) > > > > >> > >> ... 8 more > > > > >> > > > > > > >> > > > > > > >> > > Is it because new documents have something unusual in their > > > content? > > > > >> Even > > > > >> > > if that is the case, i am requesting only one field which is > > > **id** > > > > >> and > > > > >> > > there is nothing unusual about it. > > > > >> > > > > > > >> > > Any idea what what is issue here ? > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > >
