[
https://issues.apache.org/jira/browse/HADOOP-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986193#comment-17986193
]
ASF GitHub Bot commented on HADOOP-19343:
-----------------------------------------
cnauroth commented on code in PR #7761:
URL: https://github.com/apache/hadoop/pull/7761#discussion_r2167217331
##########
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopOutputStream.java:
##########
@@ -80,45 +175,191 @@ private static OutputStream
createOutputStream(GoogleCloudStorageFileSystem gcsf
String.format("'%s' already exists", gcsPath)).initCause(e);
}
OutputStream outputStream = Channels.newOutputStream(channel);
- int bufferSize = fileSystemConfiguration.getOutStreamBufferSize();
+ int bufferSize = gcsfs.getConfiguration().getOutStreamBufferSize();
return bufferSize > 0 ? new BufferedOutputStream(outputStream, bufferSize)
: outputStream;
}
@Override
public void write(int b) throws IOException {
throwIfNotOpen();
- outputStream.write(b);
+ tmpOut.write(b);
statistics.incrementBytesWritten(1);
statistics.incrementWriteOps(1);
}
@Override
public void write(@Nonnull byte[] b, int offset, int len) throws IOException
{
throwIfNotOpen();
- outputStream.write(b, offset, len);
+ tmpOut.write(b, offset, len);
statistics.incrementBytesWritten(len);
statistics.incrementWriteOps(1);
}
+ private void commitTempFile() throws IOException {
+ // TODO(user): return early when 0 bytes have been written in the temp
files
Review Comment:
I'm going to remove the "(user)" when I commit this. This is a Google
commenting standard, but it's not typical here in the Apache Hadoop codebase.
> Add native support for GCS connector
> ------------------------------------
>
> Key: HADOOP-19343
> URL: https://issues.apache.org/jira/browse/HADOOP-19343
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 3.5.0
> Reporter: Abhishek Modi
> Assignee: Arunkumar Chacko
> Priority: Major
> Labels: pull-request-available
> Attachments: GCS connector for Hadoop.pdf, Google Cloud Storage
> connector for Hadoop-1.pdf, Google Cloud Storage connector for Hadoop.pdf,
> Google Cloud Storage connector for Hadoop.v1.pdf, Google Cloud Storage
> connector for Hadoop_v1.pdf
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
