malon64 commented on PR #2185: URL: https://github.com/apache/iceberg-rust/pull/2185#issuecomment-4706025374
Hey @glitchy, I tested this branch from a downstream Rust ingestion tool (single-node, Iceberg sink). Used the public API only — live_data_files via manifest_list_reader → load_manifest → entries(), scan via table.scan().select_all(), then overwrite().add_data_files(new).delete_data_files(old). Full SCD1 upsert scenario: 3-row initial load, incoming batch with 2 updates + 1 new insert. All assertions passed, snapshot.operation = Overwrite, summary correctly reports added-data-files=1 / deleted-data-files=1. One thing worth to note: the Arrow schema passed to the Parquet writer must be derived via table.metadata().current_schema().as_ref().try_into() — a manually built ArrowSchema silently drops the field ID metadata and fails with DataInvalid => Field id N not found in struct array. Not a bug in this PR (existing writer behavior), just easy for downstream consumers to trip over. API shape works well for keyed upsert use cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
