are not going to scale like traditional
>> Solr
>> > queries.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4288194.html
>> Sent from the Solr
any shards, replicas and parallel workers are
> > needed to perform the join. #3 needs to be carefully considered because
> > MapReduce distributed joins are not going to scale like traditional Solr
> > queries.
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4288194.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
age in context:
http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4288194.html
Sent from the Solr - User mailing list archive at Nabble.com.
rote
>> > It's likely that the SortStream is the issue. With the sort function you
>> > need enough memory to sort all the tuples coming from the underlying
>> > stream. The sort stream can also be done in parallel so you can split
>> the
>> > tuples fr
ort stream can also be done in parallel so you can split the
> > tuples from col1 across N worker nodes. This will give you faster sorting
> > and apply more memory to the sort.
> >
> > Can you describe your exact use case? Perhaps we can think about a
> > different Streaming
this message in context:
http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4288116.html
Sent from the Solr - User mailing list archive at Nabble.com.
it can sort 20,000,000 docs per second. The plan is
> to
> > eventually switch to the fork/join merge sort so that you get parallelism
> > within the same worker.
> >
> >
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4288083.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
--
View this message in context:
http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4288083.html
Sent from the Solr - User mailing list archive at Nabble.com.
;
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4285202.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
tream when
there are 10M+ docs returned?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4285202.html
Sent from the Solr - User mailing list archive at Nabble.com.
http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026.html
Sent from the Solr - User mailing list archive at Nabble.com.
11 matches
Mail list logo
