The time in autocommit is in milliseconds. You are committing every second while indexing. This then causes a build-up of sucessive index readers that absorb each commit, which is probably the out-of-memory.
On Wed, Jan 13, 2010 at 10:36 AM, Minutello, Nick <nick.minute...@credit-suisse.com> wrote: > > Hi, > > I have a bit of an interesting OutOfMemoryError that I'm trying to > figure out. > > My client & Solr server are running in the same JVM (for deployment > simplicity). FWIW, I'm using Jetty to host Solr. I'm using the supplied > code for the http-based client interface. Solr 1.3.0. > > My app is adding about 20,000 documents per minute to the index - one at > a time (it is listening to an event stream and for every event, it adds > a new document to the index). > The size of the documents, however, is tiny - the total index growth is > only about 170M (after about 1 hr and the OutOfMemoryError) > At this point, there is zero querying happening - just updates to the > index (only adding documents, no updates or deletes) > After about an hour or so, my JVM runs out of heap space - and if I look > at the memory utilisation over time, it looks like a classic memory > leak. It slowly ramps up until we end up with constant FULL GC's and > eventual OOME. Max heap space is 512M. > > In Solr, I'm using autocommit (to buffer the updates) > <autoCommit> > <maxDocs>10000</maxDocs> > <maxTime>1000</maxTime> > </autoCommit> > > (Aside: Now, I'm not sure if I am meant to call commit or not on the > client SolrServer class if I am using autocommit - but as it turns out, > I get OOME whether I do that or not) > > Any suggestions/advice of quick things to check before I dust off the > profiler? > > Thanks in advance. > > Cheers, > Nick > > =============================================================================== > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > =============================================================================== > > -- Lance Norskog goks...@gmail.com
