pvary commented on PR #14728: URL: https://github.com/apache/iceberg/pull/14728#issuecomment-3601724329
Essentially, there’s a race condition between adding and dropping columns. For example, if a user does the following: 1. Creates a new schema S1 2. Sends a record R1 using S1 3. Creates a new schema S2 4. Sends a record R2 using S2 If these actions occur within a short time frame and the streams are skewed, the table could end up with either: - Schema S2, if R2 arrives later - Schema S1, if R1 arrives later Afterward, querying the table with the “old” schema becomes difficult. Additionally, users cannot revert the table to any previously created schema using DynamicSink. This behavior is consistent with the current implementation, but with column-dropping support, users might expect this capability. @Guosmilesmile: Would these restrictions impact your use cases? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
