szehon-ho commented on code in PR #9454:
URL: https://github.com/apache/iceberg/pull/9454#discussion_r1458190189
##########
core/src/main/java/org/apache/iceberg/ManifestFilterManager.java:
##########
@@ -289,13 +321,38 @@ private void invalidateFilteredCache() {
cleanUncommitted(SnapshotProducer.EMPTY_SET);
}
+ private void recordPartitionMinDataSequenceNumber(ManifestFile manifest) {
Review Comment:
Agree with above. I think there were some other attempts to do this before
too, but the concern here is that you dont want to do a lot of things in the
commit critical path (here potentially opening an unlimited number of manifest
files). Yea if its something cheaper to do like reading one partition stats
file, it may be better. Also yes the plan has always been to implement convert
eq-delete to pos deletes (which can then be cleaned up by
rewritePositionDeletes), though not sure if any progress is being made there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
