hemanthboyina commented on code in PR #15979:
URL: https://github.com/apache/iceberg/pull/15979#discussion_r3106979020
##########
core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java:
##########
@@ -559,6 +564,75 @@ private static List<PartitionStatistics>
sortStatsByPartition(
return entries;
}
+ /**
+ * Populates {@code total_record_count} for partitions where it can be
reliably computed from
+ * metadata without scanning data files.
+ *
+ * <p>Per the spec, {@code total_record_count} should stay optional (NULL
means unknown). It can
+ * be computed when a partition has no equality deletes and no V2 position
delete files — i.e.,
+ * deletes are tracked exclusively via deletion vectors (DVs) or the
partition has no deletes at
+ * all. In that case, the total is: {@code data_record_count -
position_delete_record_count}.
+ *
+ * <p>DV cardinalities are exact (each position maps to one deleted row), so
the subtraction
+ * yields a precise total without reading any data files.
+ *
+ * @param stats the computed partition statistics to enrich
+ */
+ @VisibleForTesting
+ static List<PartitionStatistics>
populateTotalRecordCount(Collection<PartitionStatistics> stats) {
+ List<PartitionStatistics> result =
Lists.newArrayListWithExpectedSize(stats.size());
+ for (PartitionStatistics partitionStats : stats) {
Review Comment:
I think for v3 tables using DVs, all partitions will have total_record_count
populated since there are no equality deletes or V2 position delete files. The
partial population occurs mainly for V2→V3 upgraded tables that still have
legacy delete files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
