amogh-jahagirdar commented on code in PR #11131:
URL: https://github.com/apache/iceberg/pull/11131#discussion_r1805121855
##########
core/src/main/java/org/apache/iceberg/ManifestFilterManager.java:
##########
@@ -185,6 +200,13 @@ List<ManifestFile> filterManifests(Schema tableSchema,
List<ManifestFile> manife
return ImmutableList.of();
}
+ // The current set of referenced manifests can be trusted if it is a
subset of the manifests
+ // being filtered. If a single referenced manifest is not in the set of
manifests being filtered
+ // this indicates that the referenced manifests are stale and cannot be
trusted.
+ Set<String> manifestLocations =
Review Comment:
These manifests are the set of manifests in the parent snapshot. I think the
check that the referenced manifests is a subset of the manifests in the parent
is sufficient. The reason is in the case of a concurrent update which removes 1
or more of the currently referenced manifests, the parent would reflect this
removal (since every retry due to a concurrent update would naturally have an
updated parent and therefore an updated view of what are the manifests
currently). Added some tests in `TestTransactions` that verify this cc @rdblue
@aokolnychyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
