pvary commented on PR #14092: URL: https://github.com/apache/iceberg/pull/14092#issuecomment-3304346560
There is no guarantee, that the WriteResult serialization would remain compatible between Iceberg releases. OTOH the manifest serialization should be backwards compatible. This is the reason why manifest is used to store the temporary data which is part of a checkpoint. I don't think we would like to change this. Also writing temporary files is costly. We need to check the cost of writing more manifest files, if we decide to go down that way. We might be better off with writing more sophisticated commit logic, and generating better commit keys (maybe add specId too) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
