b-rick opened a new pull request, #14455:
URL: https://github.com/apache/iceberg/pull/14455
Description:
This PR adds support for retaining original partition spec in rewrite data
files
Problem:
Some tables may have multiple partition schemes defined through partition
evolution, and it may be desirable to rewrite data files on existing
partitions, and also to retain their partition structures. Currently, the
rewrite data files operation always chooses either the current table partition
spec, or a partition spec the user can pass in on their own
Solution:
This change adds a configuration parameter in the rewrite data files action
that will force the action to always use the spec-id associated with the
current file. This will allow us to run rewrite data files without changing the
partition structure of the original data.
Testing
Added unit tests:
- TestRewriteDataFilesAction
- testZOrderRewriteRetainingOriginalPartitionSpecs
- testBinpackRewriteRetainingOriginalPartitionSpecs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
