karuppayya commented on code in PR #11040:
URL: https://github.com/apache/iceberg/pull/11040#discussion_r1775572057
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScan.java:
##########
@@ -194,9 +195,9 @@ protected Statistics estimateStatistics(Snapshot snapshot) {
Map<NamedReference, ColumnStatistics> colStatsMap = Collections.emptyMap();
if (readConf.reportColumnStats() && cboEnabled) {
colStatsMap = Maps.newHashMap();
- List<StatisticsFile> files = table.statisticsFiles();
- if (!files.isEmpty()) {
- List<BlobMetadata> metadataList = (files.get(0)).blobMetadata();
+ Optional<StatisticsFile> statisticsFile = statisticsFile(snapshot);
+ if (statisticsFile.isPresent()) {
+ List<BlobMetadata> metadataList = statisticsFile.get().blobMetadata();
Review Comment:
+1 to introduce the table API for retrieving the stats.
But should we do a best effort here or jsut return empty when there arent
stats for the snasphot?
We dont have a means to compare against a baseline to figure if its an
approximation, unlike dialects where it could be validated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
