JeonDaehong commented on code in PR #18997:
URL: https://github.com/apache/iceberg/pull/18997#discussion_r4599963586
##########
core/src/main/java/org/apache/iceberg/BaseFile.java:
##########
@@ -484,6 +484,11 @@ public CharSequence path() {
return filePath;
}
+ @Override
+ public String location() {
Review Comment:
`location()` used to go through `path().toString()`, so this only changes
the null case (now `null` instead of an NPE), which is what the projection test
relies on.
##########
core/src/main/java/org/apache/iceberg/ManifestReader.java:
##########
@@ -414,16 +455,13 @@ private static boolean requireStatsProjection(Expression
rowFilter, Collection<S
}
static boolean dropStats(Collection<String> columns) {
- // Make sure we only drop all stats if we had projected all stats
- // We do not drop stats even if we had partially added some stats columns,
except for
- // record_count column.
- // Since we don't want to keep stats map which could be huge in size just
because we select
- // record_count, which
- // is a primitive type.
+ // Make sure we only drop all stats if we had projected all stats.
+ // Do not drop stats when any column-level stats map was selected, except
record_count which is
+ // a primitive field rather than a per-column map.
if (columns != null && !columns.containsAll(ManifestReader.ALL_COLUMNS)) {
- Set<String> intersection = Sets.intersection(Sets.newHashSet(columns),
STATS_COLUMNS);
- return intersection.isEmpty() ||
intersection.equals(Sets.newHashSet("record_count"));
+ return Sets.intersection(Sets.newHashSet(columns),
DROPPABLE_STATS_COLUMNS).isEmpty();
Review Comment:
This gives the same result as the old check for every selection except
`column_sizes`, which wasn't in `STATS_COLUMNS`, so selecting it alone used to
drop all stats, including the column sizes themselves. Good catch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]