anuragmantri opened a new pull request, #18406: URL: https://github.com/apache/iceberg/pull/18406
SQL users can tag write snapshots with `spark.sql.iceberg.snapshot-property.*` (#14545), but snapshots from `rewrite_data_files`, `rewrite_position_delete_files`, and `rewrite_manifests` ignore it, and procedures expose no other way to set snapshot properties. This applies the session properties in `BaseSnapshotUpdateSparkAction.` Properties set through `snapshotProperty()` take precedence, as write options do over session config. Sessions that already set these properties for writes will now also tag maintenance snapshots. Takes over #15842 (credit @puchengy) and adds the commit()-path test @anoopj asked for, plus the missing docs entry. Test plan: - Rewrite data files and rewrite manifests each pick up session properties (covering both commit paths) - An explicit property overrides the session value; - `CALL rewrite_manifests` with the session property set tags the snapshot. --- AI Disclosure - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: Fully reviewed by me. - Prompt Summary: Take over #15842 for Spark 4.2: apply session-config snapshot properties to snapshot-producing Spark actions, close the commit()-path test gap, and document the session property. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
