methiakshit-plutoflume commented on issue #9504: URL: https://github.com/apache/iceberg/issues/9504#issuecomment-1919419646
The way spark understand the batch of data is by taking a diff between two snapshots. So, it is important for you to keep the history of snapshots which makes you feel comfortable to handle your streaming job errors. You can think of it similar to how offsets is for kafka, snapshot is for iceberg. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
