How is this avoided? Dennis Gearon
Signature Warning ---------------- It is always a good idea to learn from your own mistakes. It is usually a better idea to learn from others’ mistakes, so you do not have to make them yourself. from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036' EARTH has a Right To Life, otherwise we all die. --- On Thu, 10/21/10, Lance Norskog <goks...@gmail.com> wrote: > From: Lance Norskog <goks...@gmail.com> > Subject: Re: OutOfMemory and auto-commit > To: solr-user@lucene.apache.org > Date: Thursday, October 21, 2010, 9:53 PM > Yes. Indexing activity suspends until > the commit finishes, then > starts. Having both queries and indexing on the same Solr > will have > this memory problem. > > Lance > > On Thu, Oct 21, 2010 at 1:16 PM, Jonathan Rochkind <rochk...@jhu.edu> > wrote: > > If I do _not_ have any auto-commit enabled, and add > 500k documents and > > commit at end, no problem. > > > > If I instead set auto-commit maxDocs to 100000 (pretty > large number), and > > try to add 500k docs, with autocommits theoretically > happening every 100k... > > I run into an OutOfMemory error. > > > > Can anyone think of any reasons that would cause this, > and how to resolve > > it? > > All I can think of is that in the first case, my > newSearcher and > > firstSearcher warming queries don't run until the > 'document add' is > > completely done. In the second case, there are > newSearcher and firstSearcher > > warming queries happening at the same time another > process is continuing to > > stream 'add's to Solr. Although at a maxDocs of > 100000, I shouldn't (I > > think) get _overlapping_ warming queries, the warming > queries should be done > > before the next commit. I think. But nonetheless, just > the fact that warming > > queries are happening at the same time 'add's are > continuing to stream, > > could that be enough to somehow increase memory usage > enough to run into > > OOM? > > > > > > -- > Lance Norskog > goks...@gmail.com >
