RussellSpitzer commented on PR #8001: URL: https://github.com/apache/iceberg/pull/8001#issuecomment-1653404609
The issue is that writing manifests does not occur until the commit is called. If you are using inherit snapshot id the retry just rewrites the metadata.json. So you have these two timelines Concurrent commit Write manifests for group A Write manifests for group B Group A writes metadata Group B writes metadata Group A commits Group B rewrites metadata Group B commits Serial commit Write manifests for group a Groups A writes metadata Group A commits ... So if writing manifests is slow and writing metadata is fast, process two takes much much longer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
