[
https://issues.apache.org/jira/browse/HADOOP-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573124#comment-16573124
]
Hudson commented on HADOOP-15576:
---------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14724 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14724/])
HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption
(ewan.higgs: rev 2ec97abb2e93c1a8127e7a146c08e26454b583fa)
* (add)
hadoop-tools/hadoop-aws/src/main/resources/META-INF/services/org.apache.hadoop.fs.MultipartUploaderFactory
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/AbstractSTestS3AHugeFiles.java
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathHandle.java
* (add)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractMultipartUploaderTest.java
* (add)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/localfs/TestLocalFSContractMultipartUploader.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestConstants.java
* (add)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/contract/hdfs/TestHDFSContractMultipartUploader.java
* (delete)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/AbstractSystemMultipartUploaderTest.java
* (delete)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemMultipartUploader.java
* (delete)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHDFSMultipartUploader.java
* (delete)
hadoop-tools/hadoop-aws/src/main/resources/META-INF/org.apache.hadoop.fs.MultipartUploaderFactory
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/MultipartUploader.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/WriteOperationHelper.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AMultipartUploader.java
* (edit) hadoop-tools/hadoop-aws/src/test/resources/contract/s3a.xml
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/TestStagingPartitionedJobCommit.java
* (add)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractMultipartUploader.java
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystemMultipartUploader.java
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PartHandle.java
* (add)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AMultipartUploaderSupport.java
> S3A Multipart Uploader to work with S3Guard and encryption
> -----------------------------------------------------------
>
> Key: HADOOP-15576
> URL: https://issues.apache.org/jira/browse/HADOOP-15576
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2
> Reporter: Steve Loughran
> Assignee: Ewan Higgs
> Priority: Blocker
> Attachments: HADOOP-15576-005.patch, HADOOP-15576-007.patch,
> HADOOP-15576-008.patch, HADOOP-15576.001.patch, HADOOP-15576.002.patch,
> HADOOP-15576.003.patch, HADOOP-15576.004.patch
>
>
> The new Multipart Uploader API of HDFS-13186 needs to work with S3Guard, with
> the tests to demonstrate this
> # move from low-level calls of S3A client to calls of WriteOperationHelper;
> adding any new methods are needed there.
> # Tests. the tests of HDFS-13713.
> # test execution, with -DS3Guard, -DAuth
> There isn't an S3A version of {{AbstractSystemMultipartUploaderTest}}, and
> even if there was, it might not show that S3Guard was bypassed, because
> there's no checks that listFiles/listStatus shows the newly committed files.
> Similarly, because MPU requests are initiated in S3AMultipartUploader,
> encryption settings are't picked up. Files being uploaded this way *are not
> being encrypted*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
