grantatspothero opened a new pull request, #10523: URL: https://github.com/apache/iceberg/pull/10523
Context from slack thread: https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1718381807647999 > We are ingesting streaming data using a java service that does iceberg FastAppend We noticed about ~20% (YMMV) of the fastappend commit time for our usecase is spent on nonrequired cleanup operations, specifically this bit which FastAppend inherits from SnapshotProducer: https://github.com/apache/iceberg/blob/apache-iceberg-1.5.2/core/src/main/java/org/apache/iceberg/SnapshotProducer.java#L422-L439 PR skips the manifest cleanup after successful commit if no retries have occurred, as no orphaned manifests could exist if no retries have occurred. Note: we do not skip cleanup operations on failures (see: `cleanAll()`), only on successful commits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
