[
https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346107#comment-16346107
]
Steve Loughran commented on HADOOP-15040:
-----------------------------------------
good discussion in the issue; hadn't thought about this much. the v2 semantics
do seem better, probably worth switching to. At least that way s3a will fail
fast if someone drops in an older AWS JAR.
But: the new API is just a client-side change, right? It's not changed the HTTP
request at all? Otherwise other stores may suffer.
interesting to see the detail on {{GetBucketAcl}}... for that s3 diag stuff
I've been thinking of, using that sounds like a good first step probe, though a
store which doesn't support ACLs may fail there even though the store itself
works. I wouldn't use it as the normal startup check.
+[~Thomas Demoor], [~ehiggs]
> Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation
> -----------------------------------------------------------
>
> Key: HADOOP-15040
> URL: https://issues.apache.org/jira/browse/HADOOP-15040
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Priority: Major
>
> My colleagues working with Yarn log aggregation found that they were getting
> this message spammed in their logs when they used an s3a:// URI for logs
> (yarn.nodemanager.remote-app-log-dir):
> {noformat}
> getting attribute Region of com.amazonaws.management:type=AwsSdkMetrics threw
> an exception
> javax.management.RuntimeMBeanException: java.lang.NullPointerException
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at
> <snip>
> Caused by: java.lang.NullPointerException
> at com.amazonaws.metrics.AwsSdkMetrics.getRegion(AwsSdkMetrics.java:729)
> at com.amazonaws.metrics.MetricAdmin.getRegion(MetricAdmin.java:67)
> at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {noformat}
> This happens even though the aws sdk cloudwatch metrics reporting was
> disabled (default), which is a bug.
> I filed a [github issue|https://github.com/aws/aws-sdk-java/issues/1375|] and
> it looks like a fix should be coming around SDK release 1.11.229 or so.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
