amogh-jahagirdar commented on code in PR #11131:
URL: https://github.com/apache/iceberg/pull/11131#discussion_r1759609664
##########
core/src/main/java/org/apache/iceberg/ManifestFilterManager.java:
##########
@@ -304,6 +309,11 @@ private ManifestFile filterManifest(Schema tableSchema,
ManifestFile manifest) {
return manifest;
}
+ boolean hasDeletedFiles =
referencedManifestLocations.contains(manifest.path());
+ if (hasDeletedFiles) {
+ return filterManifestWithDeletedFiles(manifest, tableSchema);
+ }
Review Comment:
referencedManifests are only added to when there's a file that's deleted. If
this exists we can skip the `manifestHasDeletedFiles` method which opens up the
reader to evaluate if a file has any deletes to begin with. We can skip this
because we know absolutely there is at least a single delete!
We still need to open up the manifest reader and write out the new manifest
with the deleted entry, there can be additional criteria for the delete like by
filter which need to get evaluated. However, that seems optimal already in
`filterManifestWithDeletedFiles` since we first short circuit evaluate if the
path is deleted , and if it is we don't need to evaluate against the
partition+metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
