Hi, Even if you could import giant files, I'd avoid it because it feels like just asking for trouble. Chunk the file into smaller pieces. You can index such smaller pieces in parallel, too, and end up with faster indexing as the result.
Otis -- Solr & ElasticSearch Support http://sematext.com/ On Thu, Apr 25, 2013 at 12:10 AM, Sharmila Thapa <shar...@gmail.com> wrote: > Yes, > I have again tried to post the XML of size 2.02GB, now it throws a different > error message > <http://lucene.472066.n3.nabble.com/file/n4058825/1.png> > > While searching the cause for this error message it is found that > java's setFixedLengthStreamingMode method throws this error. > Reference:Documentation:http://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html. > So we have to limit size of the file to 16 bits i.e. 2GB. > I have also tried by setting unlimited java heap size, but does not work. > > So is there anything that can be done to support upto 6GB xml file size. If > possible I would like to try to use java -Durl to import the XML data. If > this does not work, then I will try for other alternatives as you have > suggested DIH. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/what-is-the-maximum-XML-file-size-to-import-tp4058263p4058825.html > Sent from the Solr - User mailing list archive at Nabble.com.
