hcrosse opened a new pull request, #2351: URL: https://github.com/apache/iceberg-go/pull/2351
Part of #2339. ## What Expiry cleanup now reads the metadata that the expiration was planned against, including snapshots and statistics staged earlier in the transaction. For multiple expirations, cleanup collects their files before deleting any, so a failed read deletes nothing. Cleanup also preserves files for snapshots that remain in the committed table, matching [Java's cleanup](https://github.com/apache/iceberg/blob/6275107e38cc87a7d416ef231019ceb51556f196/core/src/main/java/org/apache/iceberg/ReachableFileCleanup.java#L62-L73). ## Why Cleanup could fail after a successful commit because the base table lacked a staged snapshot or an earlier expiration had already deleted a shared manifest. Retry handling is still incomplete. If a retry rebuilds a snapshot that the transaction expires, `Commit` still returns a missing-file error after the commit succeeds. The rebuilt manifest list remains, and the other expirations skip cleanup. I will replan expiration on retry in a follow-up PR. ## Testing I added regression tests for cleanup within a transaction, retained files and read failures, and checked that they fail when the corresponding fixes are reverted. Existing `PostCommit` tests pass unchanged. The full suite, race tests for `table` and `catalog`, and lint pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
