It would be nice to have an explicit section in the doc on the topic of "Dealing with Large Result Sets" to point people to the various approaches (paging, caching, export, streaming expressions, and how to select the best one for a given use case.)
(And Joel is going to promise to update the doc for this stored field restriction, right?!) -- Jack Krupansky On Wed, Jan 20, 2016 at 9:38 AM, Joel Bernstein <joels...@gmail.com> wrote: > CloudSolrStream is available in Solr 5. The "search" streaming expression > can used or CloudSolrStream can be used in directly. > > https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions > > The export handler does not export stored fields though. It only exports > fields using DocValues caches. So you may need to re-index your data to use > this feature. > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Wed, Jan 20, 2016 at 9:29 AM, Salman Ansari <salman.rah...@gmail.com> > wrote: > > > Thanks Emir, Susheel and Jack for your responses. Just to update, I am > > using Solr Cloud plus I want to get the data completely without > pagination > > or cursor (I mean in one shot). Is there a way to do this in Solr? > > > > Regards, > > Salman > > > > On Wed, Jan 20, 2016 at 4:49 PM, Jack Krupansky < > jack.krupan...@gmail.com> > > wrote: > > > > > Yes, Exporting Results Sets is the preferred and recommended technique > > for > > > returning all documents in a collection, or even simply for queries > that > > > select a large number of documents, all of which are to be returned. It > > > uses efficient streaming rather than paging. > > > > > > But... this great feature currently does not have support for > > > distributed/SolrCloud mode: > > > "The initial release treats all queries as non-distributed requests. So > > the > > > client is responsible for making the calls to each Solr instance and > > > merging the results. > > > Using SolrJ’s CloudSolrClient as a model, developers could build > clients > > > that automatically send requests to all the shards in a collection (or > > > multiple collections) and then merge the sorted sets any way they > wish." > > > > > > -- Jack Krupansky > > > > > > On Wed, Jan 20, 2016 at 8:41 AM, Susheel Kumar <susheel2...@gmail.com> > > > wrote: > > > > > > > Hello Salman, > > > > > > > > Please checkout the export functionality > > > > > https://cwiki.apache.org/confluence/display/solr/Exporting+Result+Sets > > > > > > > > Thanks, > > > > Susheel > > > > > > > > On Wed, Jan 20, 2016 at 6:57 AM, Emir Arnautovic < > > > > emir.arnauto...@sematext.com> wrote: > > > > > > > > > Hi Salman, > > > > > You should use cursors in order to avoid "deep paging issues". > Take a > > > > look > > > > > at > > > > > https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results > > . > > > > > > > > > > Regards, > > > > > Emir > > > > > > > > > > -- > > > > > Monitoring * Alerting * Anomaly Detection * Centralized Log > > Management > > > > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > > > > > > > > > > > > On 20.01.2016 12:55, Salman Ansari wrote: > > > > > > > > > >> Hi, > > > > >> > > > > >> I am looking for a way to return all documents from a collection. > > > > >> Currently, I am restricted to specifying the number of rows using > > > > Solr.NET > > > > >> but I am looking for a better approach to actually return all > > > documents. > > > > >> If > > > > >> I specify a huge number such as 1M, the processing takes a long > > time. > > > > >> > > > > >> Any feedback/comment will be appreciated. > > > > >> > > > > >> Regards, > > > > >> Salman > > > > >> > > > > >> > > > > > > > > > > > > > > >
