Hi all,
        I am testing indexing with 2000 text documents of size 2 MB
each. These documents contain words created with random characters. I
observed that the tomcat memory usage goes on increasing slowly. I tried
by removing all the cache configuration, but still memory usage
increases. Once the memory reaches to max heap specified, commit looks
like blocked until the memory is freed. With larger documents, I see
some OOMEs
        Below are few properties set in solrconfig.xml

<mainIndex>
    <useCompoundFile>false</useCompoundFile>
    <ramBufferSizeMB>128</ramBufferSizeMB>
    <mergeFactor>25</mergeFactor>
    <maxMergeDocs>2147483647</maxMergeDocs>
    <maxFieldLength>2147483647</maxFieldLength>
    <writeLockTimeout>1000</writeLockTimeout>
    <commitLockTimeout>10000</commitLockTimeout>

   <lockType>single</lockType>
   <unlockOnStartup>false</unlockOnStartup>
</mainIndex> 
<autoCommit> 
      <maxDocs>10000</maxDocs>
      <maxTime>7000</maxTime> 
</autoCommit>
<useColdSearcher>false</useColdSearcher>
<maxWarmingSearchers>10</maxWarmingSearchers>

Where does the memory get used? And how to avoid it?

Thanks,
Siddharth

Reply via email to