RS146BIJAY opened a new issue, #12228: URL: https://github.com/apache/lucene/issues/12228
### Description **Current Issue** Currently, if segment merge/force merge fails because of disk full, IndexWriter does not clean up unreferenced files created during the current segment merge. I believe this change got introduced as a part of [this patch](https://issues.apache.org/jira/browse/LUCENE-6579). Lucene considers I/O exception because of disk full as a tragedy and does not clean up unreferenced file to avoid wasting IO/CPU cycles (which can happen due to segment merge retry filling up disk again). Issue here is these unreferenced files can be huge, and it unnecessarily continues to occupy space on the node. Further, in case the disk gets 100% filled up on a node, OpenSearch mark it as unhealthy. This is one of the major reason for node drop (unhealthy nodes) for our Cx. **Solution** Lucene should delete these unreferenced files in case segment merge fails (OpenSearch cannot handle this as Lucene closes IndexWriter in case of tragic exception). To avoid wasting IO/CPU cycles, we can change the merge policy to not allow segment merge in case enough amount of space is not available (segment merge can cause disk to get full). ### Version and environment details _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
