Re: Solrcloud export all results sorted by score

2019-10-04 Thread Edward Turner
Hi Chris, Good info, thank you for that! > What's your UI & middle layer like for this application and > eventual "download" ? I'm working in a team on the back-end side of things, where we providing a REST API that can be used by clients, which include our UI, which is a React JS based app with

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Chris Hostetter
: We show a table of search results ordered by score (relevancy) that was : obtained from sending a query to the standard /select handler. We're : working in the life-sciences domain and it is common for our result sets to : contain many millions of results (unfortunately). After users browse the

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Edward Turner
Hi Walter, Thank you also for your reply. Good to know of your experience. Roughly how many documents were you fetching? Unfortunately, it's possible that some of our users could attempt to "download" many records, meaning we'd need to make a request to Solr where rows >= 150M. A key challenge for

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Jörn Franke
Maybe you can sort later using Spark or similar. For that you don’t need a full blown cluster - it runs also on localhost. > Am 03.10.2019 um 09:49 schrieb Edward Turner : > > Hi Erick, > > Many thanks for your detailed reply. It's really good information for us to > know, and although not exa

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Edward Turner
Hi Erick, Many thanks for your detailed reply. It's really good information for us to know, and although not exactly what we wanted to hear (that /export wasn't designed to handle ranking), it's much better for us to definitively know one way or the other -- and this allows us to move forward. We'

Re: Solrcloud export all results sorted by score

2019-10-01 Thread Walter Underwood
I had to do this recently on a Solr Cloud cluster. I wanted to export all the IDs, but they weren’t stored as docvalues. The fastest approach was to fetch all the IDs in one request. First, I make a request for zero rows to get the numFound. Then I fetch numFound+1000 (in case docs were added w

Re: Solrcloud export all results sorted by score

2019-10-01 Thread Erick Erickson
First, thanks for taking the time to ask a question with enough supporting details that I can hope to be able to answer in one exchange ;). It’s a pleasure to see. Second, NP with asking on Stack Overflow, they have some excellent answers there. But you’re right, this list gets more Solr-center