I do commit once a day, millions of small docs... it takes 20 minutes in average... why OOM? I see only reduced I/O...
-----Original Message----- From: Edward Capriolo [mailto:edlinuxg...@gmail.com] Sent: August-25-09 5:35 PM To: solr-user@lucene.apache.org Subject: Re: frequency of commit when building index from scratch On Tue, Aug 25, 2009 at 5:29 PM, Bill Au<bill.w...@gmail.com> wrote: > Just curious, how often do folks commit when building their Solr/Lucene > index from scratch for index with millions of documents? Should I just wait > and do a single commit at the end after adding all the documents to the > index? > > Bill > Bill in most cases you probably cannot do one large commit as you will hit OOM. How many documents can be uncommitted is based on the size of the documents. Committing every document is slow. I have done a commit every 10,000 mostly. Results may vary. Someone might have a better answer then me.
