Re: Multiple hashJoin or innerJoin

2017-06-19 Thread Zheng Lin Edwin Yeo
Hi Joel, Take this example query with more details: innerJoin(innerJoin( search(people, q=field1a:A AND field1b:B, fl= "personId,personNme,field1a,field1b", sort="personId asc", qt="/export"), search(pets, q=field2a:A AND field2b:B, fl= "petsId,petName,field2a,field2b", sort="personId asc", q

Re: Multiple hashJoin or innerJoin

2017-06-19 Thread Joel Bernstein
These are MapReduce joins so you have to stream all the records. You definitely will not be able to stream 100 fields. So you'll need to come up with a strategy that streams the minimum number fields needed to perform the join. You can use the fetch expression to fetch additional fields following t

Re: Multiple hashJoin or innerJoin

2017-06-18 Thread Zheng Lin Edwin Yeo
Hi Joel, Yes, I have tried the hashJoin. This didn't give the timeout. If I used the /select handler, I will get 1000 records returned, and if I use the /export handler, there are too many records for the browser to display. The current count is 280,000. As I have many fields (more than 100 field

Re: Multiple hashJoin or innerJoin

2017-06-18 Thread Joel Bernstein
About the timeout error. One thing to look at is with the inner join below: innerJoin(innerJoin( search(people, q=*:*, fl="personId,name", sort="personId asc"), search(pets, q=type:cat, fl="pertsId,petName", sort="personId asc"), on="personId=petsId" ) If the left side of the join is very l

Re: Multiple hashJoin or innerJoin

2017-06-18 Thread Joel Bernstein
The search expressions don't appear to be using the /export handler. Streaming joins require the export hander because all the results that match the query need to be considered in the join. When debugging these types of multi-collection joins you need to build up the expression piece by piece. Fi

Re: Multiple hashJoin or innerJoin

2017-06-16 Thread Zheng Lin Edwin Yeo
This is the full error message from the Node for the second example, which is the following query that get stucked. innerJoin(innerJoin( search(people, q=*:*, fl="personId,name", sort="personId asc"), search(pets, q=type:cat, fl="pertsId,petName", sort="personId asc"), on="personId=petsId" )

Re: Multiple hashJoin or innerJoin

2017-06-16 Thread Zheng Lin Edwin Yeo
Hi Joel, Below are the results which I am getting. If I use this query; innerJoin(innerJoin( search(people, q=*:*, fl="personId,name", sort="personId asc"), search(pets, q=type:cat, fl="pertsId,petName", sort="personId asc"), on="personId=petsId" ), search(collection1, q=*:*, fl="collect

Re: Multiple hashJoin or innerJoin

2017-06-15 Thread Zheng Lin Edwin Yeo
Hi Joel, Yes, I got this error: {"result-set":{"docs":[{"EXCEPTION":"Invalid JoinStream - all incoming stream comparators (sort) must be a superset of this stream's equalitor.","EOF":true}]}} Ok, will try out the work around first. Regards, Edwin On 15 June 2017 at 20:16, Joel Bernstein wro

Re: Multiple hashJoin or innerJoin

2017-06-15 Thread Joel Bernstein
It looks like you are running into this bug: https://issues.apache.org/jira/browse/SOLR-10512. This not been resolved yet, but I believe there is a work around which is described in the ticket. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jun 14, 2017 at 10:09 PM, Zheng Lin Edwin Yeo wro

Re: Multiple hashJoin or innerJoin

2017-06-14 Thread Zheng Lin Edwin Yeo
I have found that this is possible, but currently I have problems if the field name to join in all the 3 collections are different. For example, if in "people" collection, it is called personId, and in "pets" collection, it is called petsId. But in "collectionId", it is called collectionName, but