OK, let's back up. _What_ is your memory error? Where is it coming from? Solr sorting or your custom code? How many documents are you returning to your custom code to sort, all of them? Some subset?
Consider re-thinking how you're doing this. In Solr 4 Beta, there's the possibility of storing all your fields and doing partial updates, that is updating just a few fields in a doc. Under the covers this stores all the data and re-indexes the entire document. That may be worth exploring... And saying "my index is 30G" isn't actually than informative, the stored data (in the *.fdt and *.fdx files in your data/index dir) store the verbatim copy of the data for any fields where ou specify 'stored="true" '. They're almost totally irrelevant for searching requirements. Solr reads all of the values for a field for _all_ the documents into memory in order to sort by that field, plus some overhead, more or less depending on what version of Solr you use (which is what?). Until you show us the stack trace, it's very hard to know much.... Best Erick On Mon, Aug 27, 2012 at 12:51 AM, ravicv <ravichandra...@gmail.com> wrote: > Hi Erick > > Thanks For reply. > > My use case is I have 50 fields to index , but at the time of index all 50 > fields data is unavailable. > So we have divided them in to 24 separate cores and indexing . we are using > shards for querying. But the problem is each core is having huge data. Total > size of all 24 cores is 30 GB. > So we wrote a custom request handler to query each core and finally merge > the results and to give final response. But some search queries responses > should be sorted based on a field. Since the response is huge while sorting > we are getting heap space errors sometimes. > > Is their any better way to handle this scenario other than this? Id so > please suggest me ... > Also how SOLR is sorting the results... > > Thanks, > Ravi > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Query-regarding-multi-core-search-tp4002847p4003400.html > Sent from the Solr - User mailing list archive at Nabble.com.
