javrasya commented on issue #9444: URL: https://github.com/apache/iceberg/issues/9444#issuecomment-1893312918
I wrote my own `S3FileIO` which uses a custom `S3InputStream` which retries when it hits a socket exception and it is all stable that way. Here is the part I modified with a very primitive retry logic in [S3InputStream.java](https://github.com/apache/iceberg/blob/8018ab844bd58dfc5fce585c0db507f606aff4a0/aws/src/main/java/org/apache/iceberg/aws/s3/S3InputStream.java#L91-L116); ```java @Override public int read() throws IOException { Preconditions.checkState(!closed, "Cannot read: already closed"); positionStream(); pos += 1; next += 1; readBytes.increment(); readOperations.increment(); try { int read = stream.read(); currentRetry = 0; return read; } catch (SocketException e) { if (currentRetry < MAX_RETRIES) { currentRetry++; LOG.debug("Reopening stream after socket exception", e); return read(); } else { throw e; } } } ``` I think Iceberg with Flink should do that out of the box. What do you think @pvary @stevenzwu ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
