zinking commented on code in PR #9454:
URL: https://github.com/apache/iceberg/pull/9454#discussion_r1462652137
##########
core/src/main/java/org/apache/iceberg/ManifestFilterManager.java:
##########
@@ -289,13 +321,38 @@ private void invalidateFilteredCache() {
cleanUncommitted(SnapshotProducer.EMPTY_SET);
}
+ private void recordPartitionMinDataSequenceNumber(ManifestFile manifest) {
Review Comment:
> I had implemented a version of this as a Spark job. I had thought at the
time to integrate it as RewriteAction, something like a finally action after
all the compact jobs are run. It may be better as its Spark job and more
scalable. #6581 @aokolnychyi should we proceed with it?
seems solving the issue, it's just there are too many such kind of actions,
I'm really not a big fan of many separate spark actions. I remember there's
another one that removed pos delete etc. can we just put these in rewrite so
that the end users can be assured that after rewrite everything returns to
normal?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
