Re: Use DIH with large xml file

2009-06-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
> > > --- On Sat, 6/20/09, Erik Hatcher wrote: > > > From: Erik Hatcher > > Subject: Re: Use DIH with large xml file > > To: solr-user@lucene.apache.org > > Date: Saturday, June 20, 2009, 6:52 PM > > How are you configuring DIH to read > > those files?

Re: Use DIH with large xml file

2009-06-20 Thread Jianbin Dai
Can DIH read item by item instead of the whole file before indexing? my biggest file size is 6GB, larger than the JVM max ram value. --- On Sat, 6/20/09, Erik Hatcher wrote: > From: Erik Hatcher > Subject: Re: Use DIH with large xml file > To: solr-user@lucene.apache.org > Da

Re: Use DIH with large xml file

2009-06-20 Thread Erik Hatcher
How are you configuring DIH to read those files? It is likely that you'll need at least as much RAM to the JVM as the largest file you're processing, though that depends entirely on how the file is being processed. Erik On Jun 20, 2009, at 9:23 PM, Jianbin Dai wrote: Hi, I hav