hemanthboyina commented on code in PR #15979:
URL: https://github.com/apache/iceberg/pull/15979#discussion_r3111581026
##########
core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java:
##########
@@ -559,6 +564,75 @@ private static List<PartitionStatistics>
sortStatsByPartition(
return entries;
}
+ /**
+ * Populates {@code total_record_count} for partitions where it can be
reliably computed from
+ * metadata without scanning data files.
+ *
+ * <p>Per the spec, {@code total_record_count} should stay optional (NULL
means unknown). It can
+ * be computed when a partition has no equality deletes and no V2 position
delete files — i.e.,
+ * deletes are tracked exclusively via deletion vectors (DVs) or the
partition has no deletes at
+ * all. In that case, the total is: {@code data_record_count -
position_delete_record_count}.
+ *
+ * <p>DV cardinalities are exact (each position maps to one deleted row), so
the subtraction
+ * yields a precise total without reading any data files.
+ *
+ * @param stats the computed partition statistics to enrich
+ */
+ @VisibleForTesting
+ static List<PartitionStatistics>
populateTotalRecordCount(Collection<PartitionStatistics> stats) {
+ List<PartitionStatistics> result =
Lists.newArrayListWithExpectedSize(stats.size());
+ for (PartitionStatistics partitionStats : stats) {
Review Comment:
Right, a V3 table can use both DVs and equality deletes. To clarify,
total_record_count is populated per-partition only when that partition has no
equality deletes and no V2 position delete files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
