On Tue, Apr 14, 2009 at 11:30 AM, Gargate, Siddharth <sgarg...@ptc.com>wrote:
> Hi all, > I am testing indexing with 2000 text documents of size 2 MB > each. These documents contain words created with random characters. I > observed that the tomcat memory usage goes on increasing slowly. I tried > by removing all the cache configuration, but still memory usage > increases. Once the memory reaches to max heap specified, commit looks > like blocked until the memory is freed. With larger documents, I see > some OOMEs > Below are few properties set in solrconfig.xml > > <mainIndex> > <useCompoundFile>false</useCompoundFile> > <ramBufferSizeMB>128</ramBufferSizeMB> > <mergeFactor>25</mergeFactor> > <maxMergeDocs>2147483647</maxMergeDocs> > <maxFieldLength>2147483647</maxFieldLength> > <writeLockTimeout>1000</writeLockTimeout> > <commitLockTimeout>10000</commitLockTimeout> > > <lockType>single</lockType> > <unlockOnStartup>false</unlockOnStartup> > </mainIndex> > <autoCommit> > <maxDocs>10000</maxDocs> > <maxTime>7000</maxTime> > </autoCommit> > <useColdSearcher>false</useColdSearcher> > <maxWarmingSearchers>10</maxWarmingSearchers> > > Where does the memory get used? And how to avoid it? > What jvm parameters are you using? Also see the following: http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr#d0e105 http://www.lucidimagination.com/blog/2009/02/09/investigating-oom-and-other-jvm-issues/ -- Regards, Shalin Shekhar Mangar.
