Re: Returning all documents in a collection

2016-01-20 Thread Joel Bernstein
The limitations of the /export handler should already be documented. Lot's of documentation still todo for Solr 6 around Streaming Expressions and some left todo on SQL. The SQL interface in Solr 6 can also select and sort entire result sets as it's built on top of the Streaming API. Joel Bernste

Re: Returning all documents in a collection

2016-01-20 Thread Jack Krupansky
It would be nice to have an explicit section in the doc on the topic of "Dealing with Large Result Sets" to point people to the various approaches (paging, caching, export, streaming expressions, and how to select the best one for a given use case.) (And Joel is going to promise to update the doc

Re: Returning all documents in a collection

2016-01-20 Thread Joel Bernstein
CloudSolrStream is available in Solr 5. The "search" streaming expression can used or CloudSolrStream can be used in directly. https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions The export handler does not export stored fields though. It only exports fields using DocValues cac

Re: Returning all documents in a collection

2016-01-20 Thread Salman Ansari
Thanks Emir, Susheel and Jack for your responses. Just to update, I am using Solr Cloud plus I want to get the data completely without pagination or cursor (I mean in one shot). Is there a way to do this in Solr? Regards, Salman On Wed, Jan 20, 2016 at 4:49 PM, Jack Krupansky wrote: > Yes, Expo

Re: Returning all documents in a collection

2016-01-20 Thread Jack Krupansky
Yes, Exporting Results Sets is the preferred and recommended technique for returning all documents in a collection, or even simply for queries that select a large number of documents, all of which are to be returned. It uses efficient streaming rather than paging. But... this great feature current

Re: Returning all documents in a collection

2016-01-20 Thread Susheel Kumar
Hello Salman, Please checkout the export functionality https://cwiki.apache.org/confluence/display/solr/Exporting+Result+Sets Thanks, Susheel On Wed, Jan 20, 2016 at 6:57 AM, Emir Arnautovic < emir.arnauto...@sematext.com> wrote: > Hi Salman, > You should use cursors in order to avoid "deep pag

Re: Returning all documents in a collection

2016-01-20 Thread Emir Arnautovic
Hi Salman, You should use cursors in order to avoid "deep paging issues". Take a look at https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results. Regards, Emir -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://semate