VABVAT commented on PR #3390: URL: https://github.com/apache/iceberg-rust/pull/3390#issuecomment-6806096474
> Thanks, this matches the Java converter. One difference left: `schemas_iter()` iterates a `HashMap`, so when two older schemas carry a dropped column with the same name but a different id or type, the one that ends up in Glue can change from one commit to the next. Java walks `metadata.schemas()` in order, so the oldest schema wins. Sorting the older schemas by id before visiting them makes it deterministic: > > ```rust > let mut schemas: Vec<_> = metadata > .schemas_iter() > .filter(|schema| schema.schema_id() != current_schema.schema_id()) > .collect(); > schemas.sort_by_key(|schema| schema.schema_id()); > ``` > > A test case for it: `legacy` as `int` in schema 0, promoted to `long` in schema 1, dropped in the current schema. Glue should get `int`. Hey @David-Banquet, Sorry for keeping this PR open, totally forgot about it. Historical schemas are now sorted by schema ID before visiting them, making it deterministic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
