[
https://issues.apache.org/jira/browse/HADOOP-11151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170475#comment-14170475
]
Arun Suresh commented on HADOOP-11151:
--------------------------------------
[~zb161], we had identified another related issue, which I suspect is what you
are facing : HADOOP-11187
The are a couple of workarounds you can try till the above bug is resolved :
# One may increase the KMS authentication token validity period to some very
high number (default is 10 hours, so by default this bug will only be
encountered after 20 hours of no communication between the NN and KMS) by
putting the following in the {{kms-site.xml}} safety valve:
{code}
<property>
<name>hadoop.kms.authentication.token.validity</name>
<value>SOME VERY HIGH NUMBER</value>
</property>
{code}
# You can switch the KMS signature secret provider to the string secret
provider by putting the following in the {{kms-site.xml}} safety valve:
{code}
<property>
<name>hadoop.kms.authentication.signature.secret</name>
<value>SOME VERY SECRET STRING</value>
</property>
{code}
> Automatically refresh auth token and retry on auth failure
> ----------------------------------------------------------
>
> Key: HADOOP-11151
> URL: https://issues.apache.org/jira/browse/HADOOP-11151
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 2.6.0
> Reporter: zhubin
> Assignee: Arun Suresh
> Fix For: 2.6.0
>
> Attachments: HADOOP-11151.1.patch, HADOOP-11151.2.patch,
> HADOOP-11151.3.patch, HADOOP-11151.4.patch, HADOOP-11151.5.patch
>
>
> Enable CFS and KMS service in the cluster, initially it worked to put/copy
> file into encryption zone. But after a while (might be one day), it fails to
> put/copy file into the encryption zone with the error
> java.util.concurrent.ExecutionException: java.io.IOException: HTTP status
> [403], message [Forbidden]
> The kms.log shows below
> AbstractDelegationTokenSecretManager - Updating the current master key for
> generating delegation tokens
> 2014-09-29 13:18:46,599 WARN AuthenticationFilter - AuthenticationToken
> ignored: org.apache.hadoop.security.authentication.util.SignerException:
> Invalid signature
> 2014-09-29 13:18:46,599 WARN AuthenticationFilter - Authentication
> exception: Anonymous requests are disallowed
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Anonymous requests are disallowed
> at
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:184)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:331)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
