amogh-jahagirdar commented on code in PR #11335:
URL: https://github.com/apache/iceberg/pull/11335#discussion_r1803860090
##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3InputStream.java:
##########
@@ -66,10 +70,14 @@ class S3InputStream extends SeekableInputStream implements
RangeReadable {
private int skipSize = 1024 * 1024;
private RetryPolicy<Object> retryPolicy =
RetryPolicy.builder()
- .handle(
- ImmutableList.of(
- SSLException.class, SocketTimeoutException.class,
SocketException.class))
- .onFailure(failure -> openStream(true))
+ .handle(RETRYABLE_EXCEPTIONS)
+ .onRetry(
+ e -> {
+ LOG.warn(
+ "Retrying read from S3, reopening stream (attempt {})",
e.getAttemptCount());
+ resetForRetry();
+ })
+ .onFailure(e -> LOG.error("Failed to read from S3 after retry
policy", e.getException()))
Review Comment:
"Failed to read from S3 input stream after exhausting all retries"
##########
aws/src/test/java/org/apache/iceberg/aws/s3/TestFlakyS3InputStream.java:
##########
@@ -49,10 +51,29 @@
public class TestFlakyS3InputStream extends TestS3InputStream {
+ private AtomicInteger resetForRetryCounter;
+
+ @BeforeEach
+ public void setupTest() {
+ resetForRetryCounter = new AtomicInteger(0);
+ }
+
+ @Override
+ S3InputStream newInputStream(S3Client s3Client, S3URI uri) {
+ return new S3InputStream(s3Client, uri) {
+ @Override
+ void resetForRetry() throws IOException {
+ resetForRetryCounter.incrementAndGet();
+ super.resetForRetry();
+ }
Review Comment:
Thanks for fixing this, this was crucial for verifying the retry behavior
actually resets the input stream. Seems like we were just getting lucky on the
tests before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
