pvary commented on issue #10765:
URL: https://github.com/apache/iceberg/issues/10765#issuecomment-2252818789
Which version of Flink do you using btw?
I see this in the logs `Committing append for checkpoint 19516` - this is
started, but can't see the corresponding `Committed {} to table: {}, branch:
{}, checkpointId {} in {} ms`
This means that Flink doesn't know if 19516 (snapshot `8203882888081487848`)
was successful or not. Let's assume it was successful behind the scenes.
So when it recovers it will find the metadata in the state for checkpoint
`19516`.
So we have to check the recovery codepath to understand what is happening.
https://github.com/apache/iceberg/blob/apache-iceberg-1.4.3/flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java#L144
`maxCommittedCheckpointId`
(https://github.com/apache/iceberg/blob/apache-iceberg-1.4.3/flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java#L493)
should return 19516 if the previous commit was successful, and the recovery
code should prevent it to be committed again.
What Catalog are you using? Is there any cache, or something which might
return wrong data for the table?
Thanks,
Peter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
