ZENOTME commented on issue #964:
URL: https://github.com/apache/iceberg-rust/issues/964#issuecomment-2654352762
Hi, I take some time to figure out the whole commit process. The whole
commit phase can be described as follows:
1. load current metadata from the catalog
2. create UpdateAction and apply them to the metadata
- When applying, there is the conflict detection process based on the
current local metadata load at step 1
- The conflict detection process is specific for the Update Action type.
e.g. FastAppend just appends data files so it doesn't have conflict detection.
- If conflict detection in the apply function fails, it means that the
table has some conflict and we can't commit. This process abort
3. If conflict detection passes, we can send the commit message to the
catalog.
- If the commit fails and the catalog returns CommitFailedException,
which means that we commit with stale metadata we can jump back to step 1 and
try to commit again.
4. If commit success, then done.
For now, we only support FastAppend. So we can complete the whole process
based on the FastAppend first and complete conflict detection when we introduce
other update actions. How do you think? @liurenjie1024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
