owenmonn opened a new issue, #18423: URL: https://github.com/apache/iceberg/issues/18423
### Apache Iceberg version 1.12.0 (latest release) ### Query engine None ### Please describe the bug 🐞 `HiveTableOperations` deletes the metadata of a table it has just created when a resent `create_table` returns `AlreadyExistsException`, which leaves a table in HMS that cannot be loaded. ## What happens `HiveTableOperations.doCommit` treats an `AlreadyExistsException` from `create_table` as a failed commit and deletes the new metadata file. That is right when another writer owns the name, but the exception can also answer our own request. `RetryingMetaStoreClient`, which `HiveClientPool` uses, resends `create_table` after a transport error. If the first request was applied and only its response was lost, the resend fails on the table the first request created. This is the create-path counterpart of #11814, where a resent `alter_table` was misread as a concurrent modification and #12637 added a commit status check. ## Reproduction 1. Register a metastore event listener whose `onCreateTable` sleeps for 5 seconds. It runs after the table is committed, so it only delays the `create_table` response. 2. Set `hive.metastore.client.socket.timeout=2s` and call `catalog.createTable`. The client times out and resends the request: ``` MetaStoreClient lost connection. Attempting to reconnect (1 of 1) after 1s. createTable org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out ``` `createTable` then fails although the table exists in HMS, and `loadTable` cannot read it: ``` org.apache.iceberg.exceptions.AlreadyExistsException: Table already exists: hivedb.repro_tbl org.apache.iceberg.exceptions.NotFoundException: Failed to open input stream for file: .../metadata/00000-<uuid>.metadata.json ``` ## Proposed fix On `AlreadyExistsException`, check whether the table now under the name has the UUID of the metadata being committed (as in trinodb/trino#31212): - Same UUID: the commit succeeded, so the files are kept. - Another table owns the name: the current behavior stays. - The check fails: the commit state is unknown, so the files are kept. ### Willingness to contribute - [x] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
