edgarRd opened a new pull request, #11335: URL: https://github.com/apache/iceberg/pull/11335
The retry policy for `S3InputStream` reads introduced in https://github.com/apache/iceberg/commit/c0d73f4ef5c16401bdfd62e1745faf2fbbf62177 is not actually re-opening the stream on each retry attempt given that it uses `onFailure` which, as per the [documentation](https://failsafe.dev/javadoc/core/dev/failsafe/PolicyBuilder.html#onFailure-dev.failsafe.event.EventListener-), it is triggered after the whole (retry) policy has failed completely and was unable to produce a successful result, i.e. all retries have been exhausted and failed. With the existing implementation, once a stream fails with something like `SSLException` due to a socked connection reset, the retries will keep failing (due to not actually reopening the stream) and therefore this won't work. The `onFailure` call does not work either because by the time `openStream` is called, the retry policy failure has already been propagated, failing the read operation. The tests also were not testing that the retry policy actually called `openStream(true)` to reset the stream, it only tested that the calls went through the retry policy a given number of times, generally always succeeding on the last one. This PR fixes the problem by using `onRetry` instead, as per the [documentation](https://failsafe.dev/javadoc/core/dev/failsafe/RetryPolicyBuilder.html#onRetry-dev.failsafe.event.EventListener-), to re-open the stream right before we attempt the read again. Also, it fixes the tests by actually checking that the stream is reset (re-opened). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
