I think the point here is the question about your use of the data. If you want to show it to the client, then you are unlikely to need details of more than 1 screenful of records (e.g. 10). When user goes to another screen, you rerun the query and specify values 11-20, etc. SOLR does not have a problem rerunning complex queries and returning different subset of results.
On the other hand, if you are not presenting this to the user directly and do need all records at once, perhaps you should not be pulling all records details from SOLR, but just use it for search. That is, let SOLR return just the primary keys of the matches and you can then send a request to a dedicated database with the list of IDs. Databases and drives are specifically designed around giving streaming results without crashing/timing-out. SOLR is a search system and is not perfect as a retrieval system or primary system of record (though it is getting there slowly). Hope this helps. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Sat, Sep 8, 2012 at 11:20 PM, neosky <neosk...@yahoo.com> wrote: > I am sorry that I can't get your point. Would you explain a little more? > I am still struggling with this problem. It seems crash by no meaning > sometimes. Even I reduce to 5000 records each time, but sometimes it works > well with 10000 per page. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Fail-to-huge-collection-extraction-tp4003559p4006399.html > Sent from the Solr - User mailing list archive at Nabble.com.
