I'm not sure why /stream is exporting faster then /export. It may be that the different approaches in the client are the reason for the difference. But the /export handler would be used in both scenarios if you specify qt=/export in the search() Streaming Expression.
Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Oct 25, 2018 at 3:07 AM Kamal Kishore Aggarwal < kkroyal....@gmail.com> wrote: > Any update on this. > > Regards > Kamal > > On Thu, Oct 18, 2018 at 11:50 AM Kamal Kishore Aggarwal < > kkroyal....@gmail.com> wrote: > > > Hi, > > > > Thanks again Joel for your reply. I have noted your suggestions. > > > > I observed one more thing while using solrj to fetch the data using > > /stream with export and direct /export. The solr QTime is almost same, > > however elapsed time(total time) to fetch response in streaming with > export > > is better than direct /export (Streaming export taking 30% less than > > /export). > > > > Is this also expected ? > > > > Regards > > Kamal Kishore > > > > > > > > On Tue, Oct 16, 2018 at 3:21 AM Joel Bernstein <joels...@gmail.com> > wrote: > > > >> Yes this is correct. But keep in mind Streaming Expression has a wide > >> range > >> of features that have nothing at all to do with the export handler. In > >> general with Streaming Expressions you want to find the functions that > get > >> the job done using the least amount of work. The /export handler is > often > >> not the best choice. You'll want to read through the various streaming > >> expressions to see if they might be more efficient for your use case. > >> > >> > >> Joel Bernstein > >> http://joelsolr.blogspot.com/ > >> > >> > >> On Mon, Oct 15, 2018 at 12:05 PM Kamal Kishore Aggarwal < > >> kkroyal....@gmail.com> wrote: > >> > >> > Hi, > >> > > >> > After I performed the test on my data, I found out that direct /export > >> and > >> > streaming expression with export, both are giving almost same response > >> > time. This was also pointed out by *Jan Høydahl* in his reply. > >> > > >> > Also, the documentation says export feature uses stream sorting > >> technique > >> > and streaming expression also uses steam technique. So, are they > >> internally > >> > works in same fashion. Please confirm. > >> > > >> > Regards > >> > Kamal Kishore > >> > > >> > > >> > > >> > On Tue, Oct 2, 2018 at 5:51 PM Kamal Kishore Aggarwal < > >> > kkroyal....@gmail.com> > >> > wrote: > >> > > >> > > Hi, > >> > > > >> > > Thanks Jan & Joel. > >> > > > >> > > Though I will evaluate the performance over my data, but based on > your > >> > > experience, which one of the two is better in performance ?. Please > >> > suggest > >> > > > >> > > Yeah, I know export does not get the data from all shards, but we > can > >> > > write code to aggregate the data from all shards. But only if export > >> is > >> > > better over stream. > >> > > > >> > > Thanks > >> > > Kamal Kishore > >> > > > >> > > > >> > > On Thu, Sep 27, 2018 at 11:04 PM Joel Bernstein <joels...@gmail.com > > > >> > > wrote: > >> > > > >> > >> The export handler does not do distributed search. So if you have a > >> > >> multi-shard collection you may have to use Streaming Expressions to > >> get > >> > >> exports from all shards. > >> > >> > >> > >> > >> > >> Joel Bernstein > >> > >> http://joelsolr.blogspot.com/ > >> > >> > >> > >> > >> > >> On Thu, Sep 27, 2018 at 4:32 AM Jan Høydahl <jan....@cominvent.com > > > >> > >> wrote: > >> > >> > >> > >> > Hi, > >> > >> > > >> > >> > Yes, you can choose which to use, it should give you about same > >> > result. > >> > >> > If you already work with the Solr search API it would be the > >> easiest > >> > for > >> > >> > you to consume /export as you don't need to learn the new syntax > >> and > >> > >> parse > >> > >> > the Tuple response. However, if you need to do stuff with the > docs > >> as > >> > >> you > >> > >> > stream them from Solr, then streaming expressions lets you enrich > >> the > >> > >> docs, > >> > >> > modify, join etc on the fly. > >> > >> > > >> > >> > PS: When the /export docs says it uses a streaming tecnique, it > >> does > >> > NOT > >> > >> > mean that it has uses the solr feature streaming expressions :) > >> > >> > > >> > >> > -- > >> > >> > Jan Høydahl, search solution architect > >> > >> > Cominvent AS - www.cominvent.com > >> > >> > > >> > >> > > 27. sep. 2018 kl. 09:07 skrev Kamal Kishore Aggarwal < > >> > >> > kkroyal....@gmail.com>: > >> > >> > > > >> > >> > > Hi, > >> > >> > > > >> > >> > > I have a requirement to fetch all data from a collection. One > >> way is > >> > >> to > >> > >> > use > >> > >> > > streaming expression and other way is to use export. > >> > >> > > > >> > >> > > Streaming expression documentation says *streaming functions > are > >> > >> designed > >> > >> > > to work with entire result sets rather then the top N results > >> like > >> > >> normal > >> > >> > > search. This is supported by the /export handler.* > >> > >> > > > >> > >> > > Also, Export handler documentation says *this feature uses a > >> stream > >> > >> > sorting > >> > >> > > technique that begins to send records within milliseconds and > >> > >> continues > >> > >> > to > >> > >> > > stream results until the entire result set has been sorted and > >> > >> exported.* > >> > >> > > > >> > >> > > These two statements concludes to me that for fetching entire > >> > results > >> > >> > > streaming expressions uses export handler and export handler > uses > >> > >> stream, > >> > >> > > so, whether I use streaming expression or export handler, they > >> are > >> > >> > > internally same and would have same performance. I am correct > >> over > >> > >> here > >> > >> > to > >> > >> > > say so ? > >> > >> > > > >> > >> > > > >> > >> > > Ref Links: > >> > >> > > > >> > >> > > > >> https://lucene.apache.org/solr/guide/6_6/streaming-expressions.html > >> > >> > > > >> https://lucene.apache.org/solr/guide/6_6/exporting-result-sets.html > >> > >> > > > >> > >> > > > >> > >> > > Regards > >> > >> > > Kamal Kishore > >> > >> > > >> > >> > > >> > >> > >> > > > >> > > >> > > >
