pvary commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1853832988
> When you say table commits, does that mean the entire table? Not the whole table data, but the table metadata. Starting from the end of the history (most recent), it goes through the history to find the most recent commit created by this job/operator > Also could there be a scenario that because of state mis allignment, that is not able to get the highest checkpointId for the jobId and operatorID It is entirely possible. In this case there are 2 possibilities: - If the Flink operator state does not contain any uncommitted data, then the job will start, and it will continue to write to the table - If the Flink operator state does contain data which is uncommitted, then the job will fail, because it could not restore the job state. > thats why it then after a restart not able to commit to a table? See above. > Do we need to add an operator uid, to make sure state compatibility? That is definitely a good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
