mattfaltyn commented on code in PR #2597:
URL: https://github.com/apache/iceberg-go/pull/2597#discussion_r4611562360
##########
table/internal/parquet_files.go:
##########
@@ -1927,17 +1927,35 @@ func resolveParquetStatsColumns(meta
*metadata.FileMetaData, statsCols map[int]S
return columns
}
+// ParquetColumnStatsPresence reads the optional fields from the footer because
+// Arrow's typed statistics can report an absent null count as zero or a
+// one-sided bound as a min/max pair.
+func ParquetColumnStatsPresence(meta *metadata.FileMetaData, rowGroup, column
int) (nullCount, bounds bool) {
+ chunk := meta.RowGroups[rowGroup].Columns[column].MetaData
Review Comment:
Thanks for the careful review! Missing stats now require an explicit
partition value when importing, and the Arrow workaround documents the
per-column encryption limitation. I added regression coverage for missing
stats, legacy and string bounds, and null partition writes. `make test` and
`make lint` passed locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]