qzyu999 commented on issue #3554: URL: https://github.com/apache/iceberg-python/issues/3554#issuecomment-4804289294
## Additional iceberg-rust Issues (Upstream Prerequisites) The following iceberg-rust issues/PRs are actively being worked on and provide the Transaction-layer primitives needed for compaction, CoW, and overwrite operations through DataFusion: ### Transaction Actions (Commit Layer) - [iceberg-rust#1607](https://github.com/apache/iceberg-rust/issues/1607) Add RewriteFiles support (umbrella, assigned to @CTTY) - [iceberg-rust#2244](https://github.com/apache/iceberg-rust/issues/2244) Implement RewriteFilesAction (sub-issue of #1607) - [iceberg-rust#2185](https://github.com/apache/iceberg-rust/pull/2185) OverwriteAction with CoW delete support (PR under review by @glitchy) - [iceberg-rust#2711](https://github.com/apache/iceberg-rust/issues/2711) DataFusion non-append InsertOp is silently committed as append (confirms the gap) ### Supporting Infrastructure - [iceberg-rust#2620](https://github.com/apache/iceberg-rust/pull/2620) MergingSnapshotProducer (draft PR by @CTTY, foundation for RewriteFiles) - [iceberg-rust#2242](https://github.com/apache/iceberg-rust/issues/2242) Process delete files when writing snapshots - [iceberg-rust#2243](https://github.com/apache/iceberg-rust/issues/2243) Implement SnapshotValidator - [iceberg-rust#2556](https://github.com/apache/iceberg-rust/issues/2556) Unify RewriteManifestsAction with SnapshotProducer ### What We Need to Contribute to iceberg-rust - **Bounded-memory session helper** utility for creating DataFusion SessionContext with FairSpillPool + spill-to-disk (no existing issue purely additive infrastructure) - **pyiceberg_core.execution module** Python FFI entry points for bounded-memory operations (extend existing indings/python/ with execution functions) - **IcebergOverwriteCommitExec** DataFusion ExecutionPlan node wiring OverwriteAction/RewriteFiles into the execution pipeline (depends on #2185 or #2244 landing first) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
