What is this huge file? Solr XML? CSV? Anyway, if it's a local file, you can get Solr to directly read/stream it via stream.file Examples in http://wiki.apache.org/solr/UpdateCSV but it should work for any update format, not just CSV.
-Yonik http://www.lucidimagination.com On Thu, Oct 8, 2009 at 6:47 AM, Pravin Karne <pravin_ka...@persistent.co.in> wrote: > Hi, > I am new to solr. I am able to index, search and update with small > size(around 500mb) > But if I try to index file with 5 to 10 or more that (500mb) it gives memory > heap exception. > While investigation I found that post jar or post.sh load whole file in > memory. > > I use one work around with dividing small file in small files..and it's > working > > Is there any other way to post large file as above work around is not > feasible for 1 TB file > > Thanks > -Pravin > > > DISCLAIMER > ========== > This e-mail may contain privileged and confidential information which is the > property of Persistent Systems Ltd. It is intended only for the use of the > individual or entity to which it is addressed. If you are not the intended > recipient, you are not authorized to read, retain, copy, print, distribute or > use this message. If you have received this communication in error, please > notify the sender and delete all copies of this message. Persistent Systems > Ltd. does not accept any liability for virus infected mails. >
