[
https://issues.apache.org/jira/browse/HADOOP-13381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HADOOP-13381:
-------------------------------
Attachment: HADOOP-13381.01.patch
The {{KeyProviderCache}} is necessary according to HDFS-7718.
To fix the issue, I can think of 2 options:
- Change the cache to recognize different clients.
- Update KMSClientProvider to favor new tokens.
I chose option #2 because #1 would increase {{KeyProvider}} object based on
client number, and even so we still need to update the tokens since after a MR
job, a token may be explicitly cancelled.
This is more of an end-to-end thing, but I tried to mimic it in TestKMS to keep
it simple. Patch 1 attached.
Thanks [~rkanter] again for helping me understand Yarn log aggregation! Ping
[~asuresh] / [~andrew.wang] for review, thank you in advance.
> KMS clients running in the same JVM should use updated KMS Delegation Token
> ---------------------------------------------------------------------------
>
> Key: HADOOP-13381
> URL: https://issues.apache.org/jira/browse/HADOOP-13381
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
> Affects Versions: 2.6.0
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Priority: Critical
> Attachments: HADOOP-13381.01.patch
>
>
> When {{/tmp}} is setup as an EZ, one may experience YARN log aggregation
> failure after the KMS token is expired. The MR job itself runs find though.
> When this happens, YARN NodeManager's log will show
> {{AuthenticationException}} with token is expire / token can't be found in
> cache, depending on whether the expired token is removed by the background or
> not.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
