Hi All, I am looking for some help to solve an out of memory issue which we are facing. We are storing messages in solr as documents. We are running a pruning job every night to delete old message documents. We are deleting old documents by calling multiple delete by id query to solr. Document count can be in millions which we are deleting using SolrJ client. We are using delete by id because it is faster than delete by query. It works great for few days but after a week these delete by id get accumulated in Linked hash map of UpdateLog (variable name as olddeletes). Once this map is full then we are seeing out of memory.
We did look into heap dump then we found that this map is storing BytesRef as key and LogPtr as value. BytesRef is the one which is taking a lot of memory. BytesRef is storing reference of all the ids which we are deleting. I am not sure why it is keeping the reference of all old deletes. I looked at solr code but I could not trace how it is cleaning this map. There is a deleteAll method in UpdateLog and only test cases are calling this method. Did anyone face the same issue? I really appreciate a reply for this problem. *Note*: We are running Solr in cloud and because of this there is high gc pause which is causing first replica go in recovery then leader and replica crashes. - Rohit
