adityaksolves commented on PR #18517: URL: https://github.com/apache/iceberg/pull/18517#issuecomment-6362988084
Hi @pvary @mxm, sorry for the delay on my side, and thanks for the detailed analysis. To answer the earlier question: I picked this up from #18499 (reported by @kuni-92) and reproduced it with a unit test in TestTableMetadataCache (DROP COLUMN -> ADD COLUMN with the same name), rather than hitting it in a production job. I went through the cases A–D from your comment. I agree that ordering alone doesn't cover A–C, since an exact match against an older schema still wins over a convertible match on the current schema, and that D (falling back to the old schema) is the intended lenient behavior. My plan is to: 1. Rebase this PR on main. 2. Adopt the "prefer a schema where every record column stays visible, even with DATA_CONVERSION_NEEDED" approach from pvary@f322d30 (keeping the credit to Peter), and 3. Add regression tests for cases A, B, C and D. Does that direction sound good to you, or would you prefer I land the ordering-only change here as a first step and do the broader fix in a follow-up PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
