Take a look at StatementImpl.constructStream() This uses a SolrStream to connect to the /sql handler. You can use the same approach to send a request to the /stream handler just by changing the parameters. Then you can open and read the SolrStream.
We don't yet have a load balancing SolrStream. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jul 26, 2016 at 11:27 AM, Timothy Potter <thelabd...@gmail.com> wrote: > Ok, makes sense now, thanks Joel. We should probably add some earlier > error checking vs. letting the code get all the way into the > HashQParser. > > So this raises a separate question that I haven't been able to figure > out, namely, do we have an example of just POST'ing the expression to > the /stream handler from SolrJ client code? i.e. I don't want to parse > and execute the streaming expression on the client side, rather, I > want to post it to the server side. Currently, my client code is a big > copy and paste of the /stream request handler (with 1 obvious omission > ;-) Specifically, I wasn't able to figure out how to parse the tuple > stream coming back using SolrJ code if I just post the expression to > /stream. > > Tim > > On Tue, Jul 26, 2016 at 8:54 AM, Joel Bernstein <joels...@gmail.com> > wrote: > > The difference would be if you are compiling and running the expression > in > > a java class or sending it to the /stream handler to be compiled. > > > > If you're compiling it and running it locally you could get this error > > because the StreamContext would not have the numWorkers variable set. > > > > The /stream handler always sets the numWorkers variable, so in theory you > > would never see this error if the /stream handler is executing the > > expression. > > > > Joel Bernstein > > http://joelsolr.blogspot.com/ > > > > On Tue, Jul 26, 2016 at 10:44 AM, Timothy Potter <thelabd...@gmail.com> > > wrote: > > > >> it's from a unit test, but not sure why that matters? If I wrap the > >> expression in a parallel expression with explicit workers=1, then it > >> works > >> > >> On Thu, Jul 21, 2016 at 11:13 AM, Joel Bernstein <joels...@gmail.com> > >> wrote: > >> > Are you getting this error from a test case you've setup or from a > manual > >> > call to the /stream handler? > >> > > >> > Joel Bernstein > >> > http://joelsolr.blogspot.com/ > >> > > >> > On Thu, Jul 21, 2016 at 12:28 PM, Timothy Potter < > thelabd...@gmail.com> > >> > wrote: > >> > > >> >> I'm working with 6.1.0 release and I have a single SolrCloud instance > >> >> with 1 shard / 1 replica. Somehow I'm triggering this, which from > what > >> >> I can see, means workers == 0, but how? Shouldn't workers default to > 1 > >> >> > >> >> I should mention that my streaming expression doesn't include any > >> >> workers, i.e. it is simply: > >> >> > >> >> val hashJoinExpr = > >> >> s""" > >> >> | hashJoin( > >> >> | search(${ratingsCollection}, > >> >> | q="*:*", > >> >> | fl="movie_id,user_id,rating", > >> >> | sort="movie_id asc", > >> >> | qt="/export", > >> >> | partitionKeys="movie_id"), > >> >> | hashed=search(${moviesCollection}, > >> >> | q="*:*", > >> >> | fl="movie_id,title", > >> >> | sort="movie_id asc", > >> >> | qt="/export", > >> >> | partitionKeys="movie_id"), > >> >> | on="movie_id" > >> >> | ) > >> >> """.stripMargin > >> >> > >> >> > >> >> > >> >> 2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase > >> >> - java.io.IOException: java.lang.RuntimeException: > >> >> java.lang.ArithmeticException: / by zero > >> >> at > >> >> > >> > org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130) > >> >> at > >> >> > >> > org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752) > >> >> at > >> >> > >> > org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735) > >> >> at > >> >> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473) > >> >> at > >> >> > >> > org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102) > >> >> at > >> org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91) > >> >> at > >> >> > >> > org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386) > >> >> at > >> >> > >> > org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064) > >> >> at > >> >> > >> > org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234) > >> >> at > >> >> > >> > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751) > >> >> at > >> >> > >> > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627) > >> >> at > >> >> > >> > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643) > >> >> at > >> >> > >> > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529) > >> >> at > >> >> > >> > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293) > >> >> at > >> >> > >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156) > >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036) > >> >> at > >> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657) > >> >> at > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464) > >> >> at > >> >> > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) > >> >> at > >> >> > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) > >> >> at > >> >> > >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) > >> >> at > >> >> > >> > org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) > >> >> at > >> >> > >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) > >> >> at > >> >> > >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) > >> >> at > >> >> > >> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) > >> >> at > >> >> > >> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) > >> >> at > >> >> > >> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) > >> >> at > >> >> > >> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > >> >> at > >> >> > >> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) > >> >> at > >> >> > >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > >> >> at > >> >> > >> > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) > >> >> at > >> >> > >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) > >> >> at org.eclipse.jetty.server.Server.handle(Server.java:518) > >> >> at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) > >> >> at > >> >> > >> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) > >> >> at > >> >> > >> > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) > >> >> at > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) > >> >> at > >> >> > >> > org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) > >> >> at > >> >> > >> > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) > >> >> at > >> >> > >> > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) > >> >> at > >> >> > >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) > >> >> at > >> >> > >> > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) > >> >> at java.lang.Thread.run(Thread.java:745) > >> >> Caused by: java.lang.RuntimeException: > java.lang.ArithmeticException: / > >> by > >> >> zero > >> >> at > >> >> > >> > org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215) > >> >> at > >> >> > >> > org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127) > >> >> ... 42 more > >> >> Caused by: java.lang.ArithmeticException: / by zero > >> >> at > >> >> > >> > org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210) > >> >> ... 43 more > >> >> > >> >
