jlkvanloon opened a new pull request, #15982: URL: https://github.com/apache/iceberg/pull/15982
### What Add support for committing a `rewriteManifests` operation to a named branch via `toBranch(String branch)`. ### Why **Manifest clustering for scan performance** In tables with many partitions, manifest files that span multiple partitions cannot be pruned during planning even after the manifest list has been read. Rewriting manifests clustered by partition (`.clusterBy(DataFile::partition)`) significantly reduces the metadata read overhead for single-partition scans. **Write-Audit-Publish (WAP) workflow** Our ingestion pipeline follows a WAP pattern: data is written to a branch, verified, and then merged into `main`. Before merging, we want to compact and cluster the branch's manifests so that `main` remains efficient after the merge. Without branch support in `rewriteManifests`, this step is not possible without merging unoptimised metadata first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
