steveloughran commented on issue #10340: URL: https://github.com/apache/iceberg/issues/10340#issuecomment-2288926718
I can't speak for the S3FileIO developers; S3AFS is where I code and while there's a lot of work there for recovery [here](https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ARetryPolicy.java) and elsewhere, we are all still finding obscure recovery failures one by one, such as how the AWS SDK doesn't recovery properly [if a multipart part upload fails with a 500](https://github.com/apache/hadoop/pull/6938). 1. If you want to use the S3FileIO: try those options. 2. If you want an S3 client which has fixes for all the failures we've hit: S3A is your friend. 3. Or you take up the PR, do your own iceberg release with it and let everyone know if it does/doesn't work. Real world pre-release testing is the way to do this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
