liurenjie1024 commented on issue #1038: URL: https://github.com/apache/iceberg-rust/issues/1038#issuecomment-2833282611
> > Maybe you would like to start working on this? > > Sounds good! Let me start an issue to track. > > One thing would like to hear your suggestion is, there're two possible implementations: > > * Have a dedicated retry layer, which wraps around all detailed catalogs > > * A detailed example, tensorflow has a retrying filesystem wrapper: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/retrying_file_system.h > * The benefit is, we won't have retry logic here and there > * Get detailed catalog (i.e. rest catalog) to handle retry > > * The benefit is: it has full knowledge whether or not to retry, and what to retry; for example, http status code is clearer than our error kind; resource exhausted error could be retried if we establish a new HTTP connection, etc. > * The potential downside is, we could replicate retry logic, or even have retry logic at different call stacks. I have concerns about adding retrying layer at catalog. Iceberg catalog is different from io layer like opendal or filesystem, whose operations are typically idempotent. For example, for the update table, if we received an timeout error, while operations in server has completed, then retrying operation is wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
