stevenzwu commented on code in PR #9998:
URL: https://github.com/apache/iceberg/pull/9998#discussion_r1533222022
##########
core/src/main/java/org/apache/iceberg/hadoop/HadoopStreams.java:
##########
@@ -185,8 +185,21 @@ public void flush() throws IOException {
@Override
public void close() throws IOException {
- stream.close();
- this.closed = true;
+ try {
+ stream.close();
+ // {@link org.apache.hadoop.fs.s3a.S3ABlockOutputStream#close()} calls
{@link
+ // org.apache.hadoop.fs.s3a.S3ABlockOutputStream#putObject()}
+ // which doesn't throw an exception when interrupted. Need to check
the interrupted flag to
+ // detect failed object upload
+ // and propagate the error up.
+ if (Thread.interrupted()
+ && "org.apache.hadoop.fs.s3a.S3ABlockOutputStream"
+ .equals(stream.getWrappedStream().getClass().getName())) {
+ throw new IOException("object upload failed as it was interrupted
during close");
+ }
+ } finally {
+ this.closed = true;
Review Comment:
Personally I am slightly favoring changing `closed` after the stream is
actually closed. I checked the existing Iceberg code on handling close. I saw
two styles: (1) setting `closed` before the action (2) just set it after the
action without finally. It is not be necessary to set it In error/failure case,
because close didn't complete successfully.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
