c-thiel opened a new pull request, #3696:
URL: https://github.com/apache/iceberg-rust/pull/3696

   ## Which issue does this PR close?
   
   No issue yet. When another commit lands on a different branch of the same 
table between loading the table and committing, `add_snapshot` refuses the new 
snapshot with a permanent `DataInvalid` error, although a reload and a new 
attempt would commit.
   
   ## What changes are included in this PR?
   
   `TableMetadataBuilder::add_snapshot` refuses a snapshot that was built on 
outdated table metadata in two cases:
   
   - Format version 2 or later: the snapshot has a parent, and its sequence 
number is not above the table's last sequence number.
   - Format version 3: the snapshot's first-row-id is below the table's 
next-row-id.
   
   A commit to another branch advances both table-wide counters without moving 
`main`, so the requirement on `main` still passes and the commit reaches these 
checks. Both errors are now `ErrorKind::CatalogCommitConflicts` with retryable 
set. `Transaction::commit` then reloads the table and retries. A catalog that 
applies commits with this builder can answer 409, which the REST spec defines 
as a failed commit the client may retry.
   
   This matches Iceberg Java 1.11 (apache/iceberg#15126), where 
`TableMetadata.Builder.addSnapshot` throws `RetryableValidationException` for 
exactly these two checks. Java describes this as a retryable validation failure 
rather than a conflict: the commit carries stale values, and a retry with 
refreshed metadata resolves it. This PR still uses `CatalogCommitConflicts`, 
for three reasons:
   
   - This crate documents `CatalogCommitConflicts` as "Catalog commit failed 
due to outdated metadata", which is this case.
   - Java's REST `CatalogHandlers` sends the exception to the client as a 
`CommitFailedException` (409).
   - iceberg-go returns its `ErrCommitFailed` for both checks.
   
   All other `add_snapshot` errors stay `DataInvalid`. Callers that matched on 
`DataInvalid` for these two errors now get `CatalogCommitConflicts`. No 
signatures change.
   
   ## Are these changes tested?
   
   Yes, with unit tests in `spec::table_metadata_builder`:
   
   - `test_sequence_number_taken_by_another_branch_is_retryable`: a commit to 
another branch raises the last sequence number. A commit to `main` that was 
built before it carries the same sequence number and gets a retryable 
`CatalogCommitConflicts`. Rebuilt with the next sequence number, the same 
commit is accepted.
   - `test_row_lineage_invalid_snapshot` now also checks that a first-row-id 
behind next-row-id gives a retryable `CatalogCommitConflicts`.
   
   ## AI Disclosure
   
   Supported by Claude Code. Reviewed and checked by me.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to