pvary commented on issue #11894: URL: https://github.com/apache/iceberg/issues/11894#issuecomment-2571365659
> it seems that the writers are not contributing to the checkpoint/savepoint data size, whereas the committer is. This is because the writers are stateless. The committers are the only stateful operator in the sink. > checkpoints should also track the temp files as well This is exactly what the checkpoint contains. If the temp files are removed then we throw an exception during the state restore. > Could it be that when I restored from a savepoint, any temp files that were written but not yet committed are essentially "lost"? This should not happen. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
