Back with a Kerberos impersonation question. The hadoop.proxyuser.*
settings are correct, at least the same settings worked on a different
cluster that doesn't require Kerberos authentication.
I can perform my action as the basic user.
When I use the same UGI code, add
user = UGI.createProxy("new user", user);
and attempt to perform the same action I get:
java.io.IOException: Failed on local exception: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "
cdhclusterqa-2-2.clouddev.snaplogic.com/10.164.199.241"; destination host
is: "cdhclusterqa-2-1.clouddev.snaplogic.com":8020;
Nothing else has changed. Literally - it's a checkbox toggle that does
nothing but conditionally call the code in blue.
Any ideas? I did a 'relogin from keytab file' with the original user -
would I need to do that after the proxy call?
(Hmm... I'm not familiar with this code but looking at the stack trace I
realize that the HDFS call is being made in a separate thread from the one
that acquired the original UGI credentials. The thread is created in a
privileged action so it has the basic information but may not have all
threadlocal information. I don't know why that decision was made. It's
suspicious... but the basic Kerberos authentication works. It's the
impersonation that's failing.)
FWIW the bottommost few exceptions are:
exc: java.io.IOException: Failed on local exception: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]; Host Details│
exc: at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776)
│
exc: at org.apache.hadoop.ipc.Client.call(Client.java:1480)
│
exc: at org.apache.hadoop.ipc.Client.call(Client.java:1407)
│
exc: at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
│
exc: at com.sun.proxy.$Proxy91.getFileInfo(Unknown Source)
│
exc: at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
│
exc: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
│
exc: at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
│
exc: at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
│
exc: at java.lang.reflect.Method.invoke(Method.java:497)
│
exc: at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
│
exc: at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
│
exc: at com.sun.proxy.$Proxy92.getFileInfo(Unknown Source)
│
exc: at
org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2113)
│
exc: at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
│
exc: at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
│
exc: at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
│
exc: at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)
│
exc: at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
│
exc: at
com.snaplogic.snap.api.fs.hdfs.HdfsUrlConnection.attemptHdfsCreate(HdfsUrlConnection.java:227)
│
exc: at
com.snaplogic.snap.api.fs.hdfs.HdfsUrlConnection.access$500(HdfsUrlConnection.java:62)
│
exc: at
com.snaplogic.snap.api.fs.hdfs.HdfsUrlConnection$3.run(HdfsUrlConnection.java:196)
│
exc: at
com.snaplogic.snap.api.fs.hdfs.HdfsUrlConnection$3.run(HdfsUrlConnection.java:191)
│
exc: at java.security.AccessController.doPrivileged(Native Method)
│
exc: at javax.security.auth.Subject.doAs(Subject.java:422)
│
exc: at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
│
exc: at
com.snaplogic.snap.api.fs.hdfs.HdfsUrlConnection.getOutputStream(HdfsUrlConnection.java:190)
│
exc: at
com.snaplogic.snap.api.binary.SimpleWriter$GetOutputStream.call(SimpleWriter.java:145)
│
exc: at
com.snaplogic.snap.api.binary.SimpleWriter$GetOutputStream.call(SimpleWriter.java:136)
│
exc: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
│
exc: at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
│
exc: at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
│
exc: at java.lang.Thread.run(Thread.java:745)
│
exc: Caused by: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]
│
exc: at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:682)
│
exc: at java.security.AccessController.doPrivileged(Native Method)
│
exc: at javax.security.auth.Subject.doAs(Subject.java:422)
│
exc: at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
│
exc: at
org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:645)
│
exc: at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:732)
│
exc: at
org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:370)
│
exc: at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
│
exc: at org.apache.hadoop.ipc.Client.call(Client.java:1446)
│
exc: ... 31 more
│
exc: Caused by: org.apache.hadoop.security.AccessControlException: Client
cannot authenticate via:[TOKEN, KERBEROS]
│
exc: at
org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:172)
│
exc: at
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:396)
│
exc: at
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:555)
│
exc: at
org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:370)
│
exc: at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:724)
│
exc: at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:720)
│
exc: at java.security.AccessController.doPrivileged(Native Method)
│
exc: at javax.security.auth.Subject.doAs(Subject.java:422)
│
exc: at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
│
exc: at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:720)
│
exc: ... 34 more
----
Bear Giles
Sr. Java Application Engineer
[email protected]
Mobile: 720-749-7876
<http://www.snaplogic.com/about-us/jobs>
*SnapLogic Inc | 929 Pearl St #200 | 80303 CO 80302 | USA*
*SnapLogic Inc | 2 W 5th Avenue 4th Floor | San Mateo CA 94402 | USA *
This message is confidential. It may also be privileged or otherwise
protected by work product immunity or other legal rules. If you have
received it by mistake, please let us know by e-mail reply and delete it
from your system; you may not copy this message or disclose its contents to
anyone. The integrity and security of this message cannot be guaranteed on
the Internet.
