I think you are describing JVM memory usage you are observing from the outside (e.g. by looking at the top). You can't really tell what's going on inside the JVM like that, though. Like Linux makes use of any available RAM for FS buffering, the JVM tends to make use of the available heap for storing objects before collecting and throwing them away.
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Jianbin Dai <djian...@yahoo.com> > To: solr-user@lucene.apache.org > Sent: Tuesday, May 26, 2009 1:36:50 PM > Subject: Re: Is it memory leaking in solr? > > > > Hi Otis, > > The slowness was due to the JVM memory limit set by tomcat.. I have solved > this > problem. Initially I thought there might be memory leaking because I noticed > the > following behavior: > In the peak of indexing, almost all 4GB of memory was used. Once indexing is > done, the memory usage was about 3GB. If I delete all indexing, and shutdown > solr, I still noticed that about 2 GB memory used, much more than the initial > memory usage about 250M. > I am not sure if I guess it right. Thanks. > > > --- On Tue, 5/26/09, Otis Gospodnetic wrote: > > > From: Otis Gospodnetic > > Subject: Re: Is it memory leaking in solr? > > To: solr-user@lucene.apache.org > > Date: Tuesday, May 26, 2009, 10:03 AM > > > > Jianbin, > > > > If you connect to that Java process with jconsole, do you > > see a lot of garbage collection activity? > > > > What makes you think there is a memory leak? The > > slowness? > > > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > > > > > ----- Original Message ---- > > > From: Jianbin Dai > > > To: solr-user@lucene.apache.org > > > Sent: Monday, May 25, 2009 1:05:43 PM > > > Subject: Re: Is it memory leaking in solr? > > > > > > > > > Again, indexing becomes extremely slow after indexed > > 8m documents (about 25G of > > > original file size). Here is the memory usage info of > > my computer. Does this > > > have anything to do with tomcat setting? Thanks. > > > > > > > > > top - 08:09:53 up 7:22, 1 user, load > > average: 1.03, 1.01, 1.00 > > > Tasks: 78 total, 2 > > running, 76 sleeping, 0 > > stopped, 0 zombie > > > Cpu(s): 49.9%us, 0.2%sy, 0.0%ni, > > 49.8%id, 0.2%wa, 0.0%hi, 0.0%si, > > 0.0%st > > > Mem: 4044776k total, 3960740k > > used, 84036k free, 42196k buffers > > > Swap: 2031608k total, > > 84k used, 2031524k free, > > 2729892k cached > > > > > > PID USER PR > > NI VIRT RES SHR S %CPU %MEM > > TIME+ COMMAND > > > > > > > > > > 3322 root 21 0 > > 1357m 1.0g 11m S 100 27.0 397:51.74 java > > > > > > > > > > > > --- On Mon, 5/25/09, Jianbin Dai wrote: > > > > > > > From: Jianbin Dai > > > > Subject: Is it memory leaking in solr? > > > > To: solr-user@lucene.apache.org, > > noble.p...@gmail.com > > > > Date: Monday, May 25, 2009, 1:27 AM > > > > > > > > I am using DIH to do indexing. After I indexed > > about 8M > > > > documents (took about 1hr40m), it used up almost > > all memory > > > > (4GB), and the indexing becomes extremely slow. > > If I delete > > > > all indexing and shutdown tomcat, it still shows > > over 3gb > > > > memory was used. Is it memory leaking? if it is, > > then the > > > > leaking is in solr indexing or DIH? > > Thanks. > > > > > > > > > > > > > > > > > > > > > > > >
