raviston commented on PR #18424:
URL: https://github.com/apache/iceberg/pull/18424#issuecomment-6938703179

   I tested this end to end against a real metastore, since the tests here use 
mocks to simulate the resend.
   
   **Reproduction:** a `MetaStoreEventListener` whose `onCreateTable` sleeps 5 
s on the first create, `hive.metastore.client.socket.timeout=2s`, and a plain 
`HiveCatalog.createTable`:
   - `main`: the metastore receives 2 `create_table` calls. `createTable` fails 
with `AlreadyExistsException`, and `loadTable` then fails with 
`NotFoundException: Failed to open input stream for file: 
.../00000-<uuid>.metadata.json`, as described in #18423.
   - This PR, rebased on current main: `createTable` succeeds and the table 
loads.
   
   `TestHiveCommits`, `TestHiveCatalog`, `TestHiveCreateReplaceTable`, 
`TestHiveViewCatalog` and `TestHiveViewCommits` pass.
   
   **One issue:** sometimes the name is taken by something that isn't an 
Iceberg table. For example, a concurrent writer creates an Iceberg view with 
that name between `loadHmsTable` and `create_table`. Then `refresh()` in 
`checkCurrentTableUuid` throws `NoSuchTableException` on every attempt, and the 
strict check retries with backoff until it returns `UNKNOWN`:
   
   | | exception | time | new metadata file |
   |---|---|---|---|
   | main | `AlreadyExistsException` | 0.5 s | deleted |
   | this PR | `CommitStateUnknownException` | 7.8 s | left behind |
   
   Something that isn't an Iceberg table can't be ours, so it can count as a 
failure:
   
   ```java
     private boolean checkCurrentTableUuid(String uuid) {
       try {
         TableMetadata refreshed = refresh();
         return refreshed != null && uuid.equals(refreshed.uuid());
       } catch (NoSuchTableException e) {
         // the name is taken by a view or a table that is not an Iceberg table
         return false;
       }
     }
   ```
   
   With this change, the view race again throws `AlreadyExistsException` and 
deletes the new metadata file. The reproduction and the test suites above still 
pass. A test like `alreadyExistsConcurrentCreate` that creates a view in 
`persistTable` would cover it.
   
   @owenmonn, the bot closed this two days ago. Could you reopen it? Without 
the fix, a lost `create_table` response makes Iceberg delete the metadata of 
the table it just created. cc @pvary, who reviewed the `alter_table` 
counterpart in #12637.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to