KazuhitoT commented on issue #269:
URL: https://github.com/apache/iceberg-python/issues/269#issuecomment-3626796757
@Fokko I have created a draft PR #2794 that implements the full set of
changes.
However, the diff is quite large, so I plan to split it into a series of
smaller PRs as follows:
1. Retry utilities
- Add `RetryConfig`, `run_with_retry`, and `run_with_suppressed_failure`
helpers.
- Unit tests for the retry utilities.
2. Transaction / Update refactoring (no behavior change)
- Introduce `_working_metadata`, `_pending_updates`, and `_StaticUpdate`
to `Transaction`.
- Add abstract `_reset_state` method and `TableRequirement.key()` to
enable replay.
- All existing tests should pass with no behavior change.
3. Transaction-level commit retry for `Update*` class
- Add retry-related properties to `TableProperties`.
- Implement `_commit_with_retry()` and `_reapply_updates()` in
Transaction.
- Implement `_reset_state()` and `_operations` for each `Update*` class
(e.g., `UpdateSchema`, `UpdateSpec`) in order to maintain and refresh the
operation states.
4. SnapshotProducer retry
- Refactor `_SnapshotProducer` to separate `apply()` and
`_build_updates()`.
- Implement retry logic for autocommit operations.
- Add `_refresh_state()` and `_reset_state()` for snapshot-specific
state.
This will likely result in 5–6 PRs.
If this overall direction looks reasonable, I would like to create the first
PR for the retry utilities.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
