FranMorilloAWS commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1855633844
I have the following issue which i believe is a bug. I have a Flink job that is consuming from one kinesis data stream and writes into multiple different Iceberg Tables. Whenever we restart/update the application as you mention the checkpointID is stored. What we are seeing is that for some tables, randomly this checkpointID is increased, which cause that for some tables no commits occur. If Flink application was currently on checkpoint 890, all tables should have stored that checkpointid, however for the tables that dont have any commits, what we see in the logs is that the checkpointid stored gets increased by almost a 100. So for the tables that correctly stored the checkpointid in state, commits happen as normal. For those that have the checkpoint increase, it wont commit until current checkpoint matches the one stored in state. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
