[GitHub] [lucene] RS146BIJAY opened a new issue, #12228: IndexWriter should clean up unreferenced files when segment merge fails

RS146BIJAY opened a new issue, #12228: URL: https://github.com/apache/lucene/issues/12228 ### Description **Current Issue** Currently, if segment merge/force merge fails because of disk full, IndexWriter does not clean up unreferenced files created during the current segment merge

[GitHub] [lucene] shaikhu commented on a diff in pull request #12201: Github 10633 Update Javadoc comment to mention gradle instead of ant

shaikhu commented on code in PR #12201: URL: https://github.com/apache/lucene/pull/12201#discussion_r1161643343 ## lucene/backward-codecs/src/test/org/apache/lucene/backward_index/TestBackwardsCompatibility.java: ## @@ -147,9 +147,10 @@ public class TestBackwardsCompatibility ex

[GitHub] [lucene] shaikhu commented on a diff in pull request #12201: Github 10633 Update Javadoc comment to mention gradle instead of ant

shaikhu commented on code in PR #12201: URL: https://github.com/apache/lucene/pull/12201#discussion_r1161643725 ## lucene/backward-codecs/src/test/org/apache/lucene/backward_index/TestBackwardsCompatibility.java: ## @@ -147,9 +147,10 @@ public class TestBackwardsCompatibility ex

[GitHub] [lucene] rmuir commented on issue #12228: IndexWriter should clean up unreferenced files when segment merge fails

rmuir commented on issue #12228: URL: https://github.com/apache/lucene/issues/12228#issuecomment-1501753122 as soon as you open a new indexwriter on the index, it will delete the unnecessary files. i don't think indexwriter should try to be a superhero and do dangerous things such as

[GitHub] [lucene] RS146BIJAY commented on issue #12228: IndexWriter should clean up unreferenced files when segment merge fails

RS146BIJAY commented on issue #12228: URL: https://github.com/apache/lucene/issues/12228#issuecomment-1501830603 @rmuir Thanks for response. For non tragic exceptions, IndexWriter [even now deletes unreferenced files](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apac

[GitHub] [lucene] rmuir commented on issue #12228: IndexWriter should clean up unreferenced files when segment merge fails

rmuir commented on issue #12228: URL: https://github.com/apache/lucene/issues/12228#issuecomment-1501874597 on a tragedy it is unsafe to delete files or anything like that. Tragic exception means just that, time to shut down. For example it can be triggered by `VirtualMachineError`, which m

[GitHub] [lucene] RS146BIJAY commented on issue #12228: IndexWriter should clean up unreferenced files when segment merge fails due to disk full

RS146BIJAY commented on issue #12228: URL: https://github.com/apache/lucene/issues/12228#issuecomment-1501907350 I agree exceptions like `VirtualMachineError` should be considered as tragic and we should avoid deleting files in those case. Actually my major concern was why segment merge fai

[GitHub] [lucene] rmuir commented on issue #12228: IndexWriter should clean up unreferenced files when segment merge fails due to disk full

rmuir commented on issue #12228: URL: https://github.com/apache/lucene/issues/12228#issuecomment-1502025869 After reviewing the original issue again, I think the current behavior is correct, it should be tragic and requires human intervention to fix so that there is enough space. Treating i

[GitHub] [lucene] vigyasharma commented on issue #12203: Scalable merge/compaction of big doc values segments.

vigyasharma commented on issue #12203: URL: https://github.com/apache/lucene/issues/12203#issuecomment-1502180974 > My basic idea is to write each field in parallel to a separate file and then perform a low-level merge of the binary data (just appending bytes to the final file). After that,

[GitHub] [lucene] kartg opened a new pull request, #12229: Change the access modifier for the "expert" readLatestCommit API to public

kartg opened a new pull request, #12229: URL: https://github.com/apache/lucene/pull/12229 ### Description The purpose of this change is to change the access modifier for the "expert" variant of the `readLatestCommit` API from package-private to public. This change also includes a uni

[GitHub] [lucene] zhaih commented on a diff in pull request #12201: Github 10633 Update Javadoc comment to mention gradle instead of ant

zhaih commented on code in PR #12201: URL: https://github.com/apache/lucene/pull/12201#discussion_r1162356564 ## lucene/backward-codecs/src/test/org/apache/lucene/backward_index/TestBackwardsCompatibility.java: ## @@ -227,7 +228,7 @@ public void testCreateMoreTermsIndex() throws