[
https://issues.apache.org/jira/browse/HADOOP-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389239#comment-17389239
]
Bobby Wang commented on HADOOP-17812:
-------------------------------------
Hi Steve,
I just found an aws account from my colleague and did the integration test, The
result can be found in attachment named [^failsafe-report.html.gz]
The auth-keys.xml I used is like that
{code:java}
<configuration>
<property>
<name>test.fs.s3a.name</name>
<value>s3a://testawss3a/</value>
</property>
<property>
<name>fs.contract.test.fs.s3a</name>
<value>${test.fs.s3a.name}</value>
</property>
<property>
<name>fs.s3a.access.key</name>
<description>AWS access key ID. Omit for IAM role-based
authentication.</description>
<value>XXXXXXX</value>
</property>
<property>
<name>fs.s3a.secret.key</name>
<description>AWS secret key. Omit for IAM role-based
authentication.</description>
<value>XXXXX</value>
</property>
<property>
<name>fs.s3a.scale.test.csvfile</name>
<value>s3a://landsat-pds/scene_list.gz</value>
</property>
<property>
<name>test.sts.endpoint</name>
<description>Specific endpoint to use for STS requests.</description>
<value>sts.amazonaws.com</value>
</property>
<property>
<name>fs.s3a.path.style.access</name>
<value>true</value>
</property>
</configuration>
{code}
The tests that failed in *ITestS3ADeleteCost* and *ITestS3ARenameCost* and
*ITestS3AFileOperationCost* are because of `DynamoDB table 'testawss3a' does
not exist in region us-west-2; auto-creation is turned off`
one test *testCustomSignerAndInitializer* failed is because of NPE
_java.lang.NullPointerException at
org.apache.hadoop.fs.s3a.auth.ITestCustomSigner$CustomSignerInitializer$StoreValue.access$200(ITestCustomSigner.java:255)
at
org.apache.hadoop.fs.s3a.auth.ITestCustomSigner$CustomSigner.sign(ITestCustomSigner.java:187)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1305)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437) at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384) at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5378) at
com.amazonaws.services.s3.AmazonS3Client.listObjectsV2(AmazonS3Client.java:970)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$listObjects$11(S3AFileSystem.java:2490)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:499)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:414) at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:377) at
org.apache.hadoop.fs.s3a.S3AFileSystem.listObjects(S3AFileSystem.java:2481) at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3720)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3583)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$MkdirOperationCallbacksImpl.probePathStatus(S3AFileSystem.java:3350)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.probePathStatusOrNull(MkdirOperation.java:135)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.getPathStatusExpectingDir(MkdirOperation.java:150)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:80) at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:45)_
and *testDistCpWithIterator* failed is because of Timeout
the detailed information can be found in the attachment.
Please help to check it. I really appreciate it.
Thx
> NPE in S3AInputStream read() after failure to reconnect to store
> ----------------------------------------------------------------
>
> Key: HADOOP-17812
> URL: https://issues.apache.org/jira/browse/HADOOP-17812
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.2.2, 3.3.1
> Reporter: Bobby Wang
> Priority: Major
> Labels: pull-request-available
> Attachments: failsafe-report.html.gz, s3a-test.tar.gz
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> when [reading from S3a
> storage|https://github.com/apache/hadoop/blob/rel/release-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L450],
> SSLException (which extends IOException) happens, which will trigger
> [onReadFailure|https://github.com/apache/hadoop/blob/rel/release-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L458].
> onReadFailure calls "reopen". it will first close the original
> *wrappedStream* and set *wrappedStream = null*, and then it will try to
> [re-get
> *wrappedStream*|https://github.com/apache/hadoop/blob/rel/release-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L184].
> But what if the previous code [obtaining
> S3Object|https://github.com/apache/hadoop/blob/rel/release-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L183]
> throw exception, then "wrappedStream" will be null.
> And the
> [retry|https://github.com/apache/hadoop/blob/rel/release-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L446]
> mechanism may re-execute the
> [wrappedStream.read|https://github.com/apache/hadoop/blob/rel/release-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L450]
> and cause NPE.
>
> For more details, please refer to
> [https://github.com/NVIDIA/spark-rapids/issues/2915]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
