pvary commented on PR #16249: URL: https://github.com/apache/iceberg/pull/16249#issuecomment-4518332066
> This change resolves the input only against the current table schema. This is not good. One of the goal of the DynamicSink was to allow schema evolution for the tables. If we don't allow matching to old schema, then we can't transition smoothly between the schema versions. The issue you are facing is highlighted in the doc: https://iceberg.apache.org/docs/latest/flink-writes/#schema-evolution > If the field was to re-appear as part of a new schema, an entirely new column would be added, which apart from the name, has nothing in common with the old column, i.e. queries for the new column will never return data of the old column. Maybe we could improve by changing the order so we try to match the latest schema first, and try to match the schemas in a different direction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
