viliam-durina commented on issue #14334: URL: https://github.com/apache/lucene/issues/14334#issuecomment-2771736474
>every application everywhere and everytime would need to call fsync on close Yeah, I don't know what would be the use case for not fsyncing before closing. Maybe if you don't care if the file is written or not, but then why are you writing it in the first place? If you don't fsync, it's like submitting an async operation, but not checking the future's result. It will mostly succeed, but if not, you won't know. >If a temporary file in the lucene index is corrupted then lucene will bail out Are we sure that we can detect every possible corruption in the file? E.g. if the file size is 0, but there should have been some data? If so then yes, the commit will crash, which is better than producing incorrect index. Only the exception will be misleading: instead of an IOException you'll have unexpected EOFExceptions, buffer overflows, array indexes out of bounds, invalid values etc. Checksum violation is the best of these. And you won't be able to reproduce any of these errors in a test. It's unlikely that you'll think that there was a hidden IO failure. The unresolved issue #10906 is also a result of a missing fsync IMO. For temporary files, we should either fsync before closing, or start reading without closing the file. The latter is more complicated, but has no performance hit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
