adityaksolves commented on PR #18517: URL: https://github.com/apache/iceberg/pull/18517#issuecomment-6234059328
Thanks @mxm, the external type-change scenario is a good point: failing the job there seems worse than writing with the old schema, and I agree we can’t tell old records from re-create requests in the dynamic sink case. Unless @pvary has a use-case that ordering alone doesn’t cover, I’ll drop the skip-filter and keep only the ordering (newest schema first, so the re-created field takes precedence). I’ll also add a test asserting the ordering. One question on ordering: should it be strictly by schema id descending, or current schema first and then the rest descending? After a rollback to an older schema, the current schema is not necessarily the one with the highest id. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
