Hi guys, I recently upgraded from Solr 4.0 to 4.8.1. I start it with a clean index (we did some change in the Solr schema in the meantime) and after some time of indexing a very big database my instance is becoming totally unusable with 99% of the heap filled. Then when I restart it it get stuck very quickly with the same memory issue so it seems linked to the size of the Lucene index more that the time spend indexing data.
Youtkit is telling me that "Lucene Merge Thread #1" (ConcurrentMergeScheduler$MergeThread) is keeping 4,095 instances of org.apache.lucene.codecs.lucene42.Lucene42DocValuesProducer$2 in some local variable(s) which retain about 422MB of RAM and keep adding more of those to the heap. Whatever this thread is doing (ConcurrentMergeScheduler javadoc is not very detailed) it does not seems to be doing it in a very streamed fashion (if not "simply" a memory leak in 4.8.1). Any idea if this is expected ? Do I have a way to control the size of the heap this thread is going to need ? I have the heap dump if anyone want more details or want to look at it. The Solr setup can be seen on https://github.com/xwiki/xwiki-platform/tree/xwiki-platform-6.2/xwiki-platform-core/xwiki-platform-search/xwiki-platform-search-solr/xwiki-platform-search-solr-api/src/main/resources/solr/xwiki/conf I know all that is only talking about Lucene classes but since on my side what I use is Solr I tough it was better to ask on this mailing list. Thanks, -- Thomas
