rdblue commented on code in PR #18816:
URL: https://github.com/apache/iceberg/pull/18816#discussion_r4622436132
##########
core/src/main/java/org/apache/iceberg/V4ManifestReader.java:
##########
@@ -463,12 +584,15 @@ Builder scanMetrics(ScanMetrics newScanMetrics) {
}
V4ManifestReader build() {
- if (scanPlanning && requestedStatsFieldIds == null) {
+ if (mode != Mode.DEFAULT && requestedStatsFieldIds == null) {
// only return stats that were requested
requestedStatsFieldIds = ImmutableSet.of();
}
Map<Integer, Evaluator> partitionFilters = projectFilters();
+ boolean changes = mode == Mode.CHANGES;
+ Tracking tracking = changes ? manifest.tracking() : null;
Review Comment:
I think the logic in this method is hard to follow.
This sets `tracking` when in changes mode. Below, `resolveChangeSnapshotId`
is called with only `tracking` as an argument, also only in changes mode. The
reason for not just passing `manifest.tracking()` directly is the other lines
that use `tracking` and create bitmap instances -- I think we only want to do
that work if the bitmaps will be used and `changes` is true, which is guarantee
with a non-null `tracking`. Then `tracking` might be null, which works for the
check on the bitmap lines, but doesn't work for `resolveChangeSnapshotId`
because that method goes directly to `manifest` when its argument is null.
This is too convoluted.
This `manifest` is actually passed into the `V4ManifestReader`, so the
bitmaps don't need to be passed separately at all. That simplifies both the
constructor and the builder. You also don't need to pass whether the status is
deleted. That can also go in the constructor.
```java
if (changes && manifest.tracking() != null) {
this.deletedPositions =
MumblingBitmaps.read(manifest.tracking().deletedPositions());
this.replacedPositions =
MumblingBitmaps.read(manifest.tracking().replacedPositions());
this.isManifestDeleted = manifest.tracking().status() ==
EntryStatus.DELETED;
this.isManifestReplaced = manifest.tracking().status() ==
EntryStatus.DELETED;
} else {
this.deletedPositions = null;
this.replacedPositions = null;
this.isManifestDeleted = false;
this.isManifestReplaced = false;
}
```
I'm also questioning whether you need two booleans if `uncommitted` and
`changes` are not allowed together. That's an okay assumption and since this is
private, you could just pass a mode through instead of adding another boolean.
Last, I think you do need the snapshot ID to find changes. But that method
should either be static and accept `ManifestFile` or should just use `manifest`
directly. It doesn't make sense to me that you would pass in half of the state
on which the method depends.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]