mukund-thakur commented on issue #16418: URL: https://github.com/apache/iceberg/issues/16418#issuecomment-4733984480
> I think RewriteTablePathSparkAction should be safely executed repeatedly to support incremental data migration. Thinking more about this, there is nothing incremental going on here. we are always overwriting. There is no checkpointing and resumable progress. So the only issue here is we are avoiding FileAlreadyExistsException in case staging location is reused. I am not sure if it is worth designing a solution which is truly incremental. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
