wombatu-kun opened a new pull request, #16943: URL: https://github.com/apache/iceberg/pull/16943
Closes #16942. Part of #16232. A replace transaction commits metadata built when it started, so on `BaseMetastoreCatalog` catalogs (Hive, Glue, JDBC, Nessie, Hadoop) changes committed in between are lost: another writer's snapshot drops out of history, and a snapshot that `expire_snapshots` removed comes back. When the table changed, the replacement is now rebuilt on the refreshed metadata and the transaction's updates are replayed, which also keeps concurrent property changes, as REST does. If the rebuilt schema, spec, sort order or format version would differ from the transaction's (IDs included), or an update cannot be replayed (for example a merge-schema `CREATE OR REPLACE`), the commit falls back to today's behavior. REST keeps merging server-side; path-based `HadoopTables` is a follow-up. Builds on #16975, #15092 and #15904. Test plan: each `CatalogTests` suite checks that a replace keeps a concurrent snapshot, does not bring back an expired snapshot, and, when a no-op schema update blocks the replay, keeps its own data and every manifest list; `TestReplaceTransaction` checks the history and that the replace's files keep their sort order, on format versions 1 to 4. The history tests fail on `main` for metastore catalogs, the replay test fails without the fallback or the cleanup filter, and the sort-order test fails without the sort-order check. --- **AI Disclosure** - Model: Claude Opus 4.8, Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: fully reviewed - Prompt Summary: Fix #16942 (part of #16232) by rebuilding a replace transaction on refreshed metadata when the table changed concurrently. The fix, its tests and this description are generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
