hsingh574 commented on issue #1382: URL: https://github.com/apache/iceberg-rust/issues/1382#issuecomment-2946484690
Hi @liurenjie1024 , this might be undesirable since its not how stateful catalogs will work, but from how the MemoryCatalog looks today it seems possible to never have a commit conflict, only requirement failures. This is because of the mutex in namespace_state, which we can acquire during `update_table` to ensure all updates commit serially and never have to conflict. So the `update_table` implementation roughly becomes: - acquire mutex - load table - assert requirements - apply updates - write new file - update metadata location - release mutex So in the example above if thread1 and thread2 try to commit concurrently, the loser will wait for the winner to commit, then acquire the lock and reload the table to see the winner's commit, on top of which they can apply their own requirements and updates. Again, this may not be desirable because this pessimistic concurrency is opposite of Iceberg's normal optimistic concurrency. Anyways, I'd like to contribute to this functionality if possible. Which piece would you suggest that has least conflict with the other open PRs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
