gtrettenero opened a new pull request, #17490: URL: https://github.com/apache/iceberg/pull/17490
`SnapshotChanges` reads manifests with no projection and copies added files with `entry.file().copy()`, so the per-column statistics maps (`column_sizes`, `value_counts`, `null_value_counts`, `nan_value_counts`, `lower_bounds`, `upper_bounds`) are always decoded and retained. Callers that only need file identity — path, size, partition, spec — pay for stats they never read, which on wide tables dominates the retained heap of the cached file lists. The removed path already uses `copyWithoutStats()`. Adds `SnapshotChanges.Builder#includeColumnStats(boolean)`, defaulting to `true`. When `false`, manifests are read with `BaseScan.scanColumns(manifest.content())` — the same curated projection `PartitionsTable` and `PartitionStatsHandler` already use for direct manifest reads — and files are copied without stats. Selecting by manifest content type keeps `content`, `referenced_data_file`, `content_offset`, `content_size_in_bytes` and `equality_ids` on delete files; a single caller-supplied column list would not be correct for both manifest types. The default maps to `ManifestReader.ALL_COLUMNS`, which `Schema#select` short-circuits, so existing callers are unaffected. The polarity is inverted relative to `Scan#includeColumnStats()`, which is opt-in because scans default to dropping stats. ### Testing Four tests in `TestSnapshotChanges` cover the default, the opt-out, partition values on a partitioned table, and `content`/`equalityFieldIds` on delete files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
