The search expressions don't appear to be using the /export handler. Streaming joins require the export hander because all the results that match the query need to be considered in the join.
When debugging these types of multi-collection joins you need to build up the expression piece by piece. First simply run the searches individually and see how long they take to fully export. You can do this by using curl to run the search expressions and saving all the records to a file. Then run a single join using curl and save the records to file. Once you get that working try the three joins together. You'll use this same approach when improving performance of the join. You look at the performance of each part of expression and improve performance in places where the expression is slow. Joel Bernstein http://joelsolr.blogspot.com/ On Sat, Jun 17, 2017 at 12:00 AM, Zheng Lin Edwin Yeo <edwinye...@gmail.com> wrote: > This is the full error message from the Node for the second example, which > is the following query that get stucked. > > innerJoin(innerJoin( > search(people, q=*:*, fl="personId,name", sort="personId asc"), > search(pets, q=type:cat, fl="pertsId,petName", sort="personId asc"), > on="personId=petsId" > ), > search(collection1, q=*:*, fl="collectionId,collectionName", > sort="collectionId asc"), > )on="personId=collectionId" > > > ------------------------------------------------------------ > ------------------------------------------------------ > Full error message: > > java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout > expired > : 50000/50000 ms > at > org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlo > ckingCallback.java:219) > at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:220) > at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:496) > at > org.apache.commons.io.output.ProxyOutputStream.write(ProxyOutputStrea > m.java:90) > at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) > at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282) > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) > at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207) > at org.apache.solr.util.FastWriter.flush(FastWriter.java:140) > at org.apache.solr.util.FastWriter.write(FastWriter.java:54) > at > org.apache.solr.response.JSONWriter.writeStr(JSONResponseWriter.java: > 482) > at > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWrit > er.java:132) > at > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559 > ) > at > org.apache.solr.handler.ExportWriter$StringFieldWriter.write(ExportWr > iter.java:1445) > at > org.apache.solr.handler.ExportWriter.writeDoc(ExportWriter.java:302) > at > org.apache.solr.handler.ExportWriter.lambda$writeDocs$4(ExportWriter. > java:268) > at > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java: > 547) > at > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWrit > er.java:193) > at > org.apache.solr.response.JSONWriter$1.add(JSONResponseWriter.java:532 > ) > at > org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:267) > > at > org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java: > 219) > at > org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter. > java:523) > at > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWrit > er.java:175) > at > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559 > ) > at > org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java: > 219) > at > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java: > 547) > at > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWrit > er.java:193) > at > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:559 > ) > at > org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java > :217) > at > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java: > 547) > at org.apache.solr.handler.ExportWriter.write( > ExportWriter.java:215) > at org.apache.solr.core.SolrCore$3.write(SolrCore.java:2564) > at > org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(Q > ueryResponseWriterUtil.java:49) > at > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java: > 809) > at org.apache.solr.servlet.HttpSolrCall.call( > HttpSolrCall.java:538) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte > r.java:347) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte > r.java:298) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet > Handler.java:1691) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java > :582) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j > ava:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav > a:548) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl > er.java:226) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl > er.java:1180) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java: > 512) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle > r.java:185) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle > r.java:1112) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j > ava:141) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont > extHandlerCollection.java:213) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl > ection.java:119) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper > .java:134) > at > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandle > r.java:335) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper > .java:134) > at org.eclipse.jetty.server.Server.handle(Server.java:534) > at org.eclipse.jetty.server.HttpChannel.handle( > HttpChannel.java:320) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav > a:251) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstra > ctConnection.java:273) > at org.eclipse.jetty.io.FillInterest.fillable( > FillInterest.java:95) > at > org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoin > t.java:93) > at > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeP > roduceConsume(ExecuteProduceConsume.java:303) > at > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceC > onsume(ExecuteProduceConsume.java:148) > at > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Exec > uteProduceConsume.java:136) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo > l.java:671) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool > .java:589) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: > 50000/50 > 000 ms > at > org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:16 > 6) > at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:51 > 1) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask. > access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask. > run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. > java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor > .java:617) > ... 1 more > > Regards, > Edwin > > > On 17 June 2017 at 11:53, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > wrote: > > > Hi Joel, > > > > Below are the results which I am getting. > > > > If I use this query; > > > > innerJoin(innerJoin( > > search(people, q=*:*, fl="personId,name", sort="personId asc"), > > search(pets, q=type:cat, fl="pertsId,petName", sort="personId asc"), > > on="personId=petsId" > > ), > > search(collection1, q=*:*, fl="collectionId,collectionName", > > sort="collectionId asc"), > > )on="petsId=collectionId" > > > > I will get this exception error. > > > > {"result-set":{"docs":[{"EXCEPTION":"Invalid JoinStream - all incoming > stream comparators (sort) must be a superset of this stream's > equalitor.","EOF":true}]}} > > > > > > > > But if I use this query: > > > > innerJoin(innerJoin( > > search(people, q=*:*, fl="personId,name", sort="personId asc"), > > search(pets, q=type:cat, fl="pertsId,petName", sort="personId asc"), > > on="personId=petsId" > > ), > > search(collection1, q=*:*, fl="collectionId,collectionName", > > sort="collectionId asc"), > > )on="personId=collectionId" > > > > The query will get stuck, until I get this message. After which, the > whole > > Solr is hanged, and I have to restart Solr to get it working again. This > is > > in Solr 6.5.1. > > > > 2017-06-17 03:16:00.916 WARN (zkCallback-8-thread-4- > > processing-n:192.168.0.1:8983_solr x:collection1_shard1_replica1 > s:shard1 > > c:collection1 r:core_node1-EventThread) [c:collection1 s:shard1 > > r:core_node1 x:collection1_shard1_replica1] o.a.s.c.c.ConnectionManager > Our > > previous ZooKeeper session was expired. Attempting to reconnect to > recover > > relationship with ZooKeeper... > > > > > > Regards, > > Edwin > > > > > > On 15 June 2017 at 23:36, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > > wrote: > > > >> Hi Joel, > >> > >> Yes, I got this error: > >> > >> {"result-set":{"docs":[{"EXCEPTION":"Invalid JoinStream - all incoming > stream comparators (sort) must be a superset of this stream's > equalitor.","EOF":true}]}} > >> > >> > >> Ok, will try out the work around first. > >> > >> Regards, > >> Edwin > >> > >> > >> On 15 June 2017 at 20:16, Joel Bernstein <joels...@gmail.com> wrote: > >> > >>> It looks like you are running into this bug: > >>> https://issues.apache.org/jira/browse/SOLR-10512. This not been > resolved > >>> yet, but I believe there is a work around which is described in the > >>> ticket. > >>> > >>> Joel Bernstein > >>> http://joelsolr.blogspot.com/ > >>> > >>> On Wed, Jun 14, 2017 at 10:09 PM, Zheng Lin Edwin Yeo < > >>> edwinye...@gmail.com> > >>> wrote: > >>> > >>> > I have found that this is possible, but currently I have problems if > >>> the > >>> > field name to join in all the 3 collections are different. > >>> > > >>> > For example, if in "people" collection, it is called personId, and in > >>> > "pets" collection, it is called petsId. But in "collectionId", it is > >>> called > >>> > collectionName, but it won't work when I place it this way below. Any > >>> > suggestions on how I can handle this? > >>> > > >>> > innerJoin(innerJoin( > >>> > search(people, q=*:*, fl="personId,name", sort="personId asc"), > >>> > search(pets, q=type:cat, fl="pertsId,petName", sort="personId > asc"), > >>> > on="personId=petsId" > >>> > ), > >>> > search(collection1, q=*:*, fl="collectionId,collectionName", > >>> > sort="personId asc"), > >>> > )on="personId=collectionId" > >>> > > >>> > > >>> > Regards, > >>> > Edwin > >>> > > >>> > On 14 June 2017 at 23:13, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > >>> > wrote: > >>> > > >>> > > Hi, > >>> > > > >>> > > I'm using Solr 6.5.1. > >>> > > > >>> > > Is it possible to have multiple hashJoin or innerJoin in the query? > >>> > > > >>> > > An example will be something like this for innerJoin: > >>> > > > >>> > > innerJoin(innerJoin( > >>> > > search(people, q=*:*, fl="personId,name", sort="personId asc"), > >>> > > search(pets, q=type:cat, fl="personId,petName", sort="personId > >>> asc"), > >>> > > on="personId" > >>> > > ), > >>> > > search(collection1, q=*:*, fl="personId,personName", > sort="personId > >>> > > asc"), > >>> > > ) > >>> > > > >>> > > Regards, > >>> > > Edwin > >>> > > > >>> > > >>> > >> > >> > > >
