hcrosse opened a new pull request, #2354: URL: https://github.com/apache/iceberg-go/pull/2354
Closes #2339. Stacked on #2351, the first PR for this issue. Its commit is included until it merges. Please review only the second commit. ## What On retry or resume, transactions with `ExpireSnapshots` replay staged changes in order against the refreshed table. Each expiry is replanned at its original position with its original options and cutoff. An append combined with an expiry can now rebase onto a peer's append. After a peer rollback or ref change, replanning retains snapshots that are reachable again. Each attempt asserts the refs that its expiry plans read against that attempt's base table, rather than pinning them across retries. Explicit `AssertRefSnapshotID` and `RollbackToSnapshot` still fail if their refs move. Cleanup uses the updates from the successful attempt, including rebuilt snapshots that the same transaction expires. If replay leaves no updates, `Commit` returns the refreshed table without another catalog commit. Transactions containing only expiries that staged no updates still make no catalog call. Transactions without an expiry keep the existing retry behavior. A retry returns `ErrTransactionUnusable` if replanning changes whether a non-main commit branch is removed, GC is disabled after refresh, or a staged update no longer applies. Only the branch removal case also matches `ErrCommitFailed`, because a new transaction can plan consistently from the refreshed table. Known limitations remain: a concurrently created ref can point to a snapshot that the expiry removes, and a concurrent GC disable is not detected without another conflict. `TableCommit` includes the ref assertions but still neither replays nor cleans up. ## Why Expiration pins currently reject concurrent commits that could be handled by replanning. Dropping the pins without replacing the stale expiration plan would be unsafe. Go now matches Java's [ordered replay after refresh](https://github.com/apache/iceberg/blob/6275107e38cc87a7d416ef231019ceb51556f196/core/src/main/java/org/apache/iceberg/BaseTransaction.java#L439-L454) and [fixed cutoff](https://github.com/apache/iceberg/blob/6275107e38cc87a7d416ef231019ceb51556f196/core/src/main/java/org/apache/iceberg/RemoveSnapshots.java#L84-L102). Unlike Go, Java [adds no requirement for snapshot removal](https://github.com/apache/iceberg/blob/6275107e38cc87a7d416ef231019ceb51556f196/core/src/main/java/org/apache/iceberg/UpdateRequirements.java#L92-L115) and has no equivalent check for a changed decision to remove the commit branch. ## Testing I added regression tests for replay, ref assertions, failure handling and cleanup, plus SQLite end-to-end tests for concurrent appends, rollbacks and branch changes. Each new test fails when its corresponding behavior is reverted. The full suite, race tests for `table` and `catalog`, and lint pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
