alexjbuck opened a new pull request, #1940:
URL: https://github.com/apache/iceberg-rust/pull/1940
Add the ability to create a tag reference atomically in the same transaction
that creates a new
snapshot via fast_append.
This adds a `with_tag()` method to `FastAppendAction` that allows specifying
a tag name. When set,
the tag will be created pointing to the newly created snapshot, all within a
single atomic catalog
update.
Example usage:
```rust
let tx = Transaction::new(&table);
let action = tx
.fast_append()
.add_data_files(data_files)
.with_tag("v1.0.0");
let tx = action.apply(tx)?;
let table = tx.commit(&catalog).await?;
```
## Which issue does this PR close?
- Closes #1939
## What changes are included in this PR?
This change adds a method to the `FastAppendAction` that allows atomic
creation of a snapshot
reference to a specified tag in the same transaction that creates the
snapshot. This enables
atomic setting of tags and appending data in a single transaction.
Without the atomic transaction guarantees, it is possible for other
processes to add new snapshots
and cleanup old snapshots (including the one _we_ just created) before a tag
can be added to the
snapshot, thus protecting it from most automated snapshot expiration
policies.
With the atomic guarantees, we can guarantee that either our data was not
committed into the table,
or it was _and_ it has a tagged snapshot reference that is protected from
expiry.
## Are these changes tested?
I created a unit test `test_fast_append_with_tag` in the
`src/transaction/append.rs` module that
verifies that there are 3 actions in the `FastAppendAction` when called with
`.with_tag("tag")`, and
that those 3 actions are a `TableUpdate::AddSnapshot` and two
`TableUpdate::SetSnapshotRef` actions,
with one setting the `MAIN_BRANCH` tag, and the second setting the tag
specified in
`.with_tag("tag")`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
