[
https://issues.apache.org/jira/browse/HADOOP-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000519#comment-16000519
]
Steve Loughran commented on HADOOP-14350:
-----------------------------------------
OK, I think right now you are going to have to help us deal with this as you
are the one with the IBM JDK. Know that Kerberos is one area where the
implementations vary between JDK implementations and versions, and that because
we need more detailed APIs than the public ones, it's fairly low level and
brittle. We are [scared of
kerberos|https://www.gitbook.com/book/steveloughran/kerberos_and_hadoop/details]
Looking at the Hadoop code when a keytab is passed it, we're assuming its
absolute:
{code}
private static String prependFileAuthority(String keytabPath) {
return keytabPath.startsWith("file://") ? keytabPath
: "file://" + keytabPath;
}
{code}
I *suspect* that this could be a cause, but it makes me worry that changing
this will break other things everywhere else -and that the use of URLs implies
that the Kerberos code expects absolute paths.
Can't you just set it up with an absolute path? I don't even see a relative
file:local.keytab ref working, as that would only confuse the condition.
Otherwise, with Hadoop 2.8 you can now run
[[KDiag|http://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/SecureMode.html#Troubleshooting_with_KDiag]
which tries to make sense of what's going on. Can you run that with all the
logging on and see if it helps?
> Relative path for Kerberos keytab is not working on IBM JDK
> -----------------------------------------------------------
>
> Key: HADOOP-14350
> URL: https://issues.apache.org/jira/browse/HADOOP-14350
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, security
> Affects Versions: 2.7.3
> Environment: IBM JDK
> Reporter: Wen Yuan Chen
>
> For the sample code below:
> public class TestKrb {
> public static void main(String[] args) throws IOException {
> String user = args[0], path = args[1];
> UserGroupInformation ugi =
> UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
> System.out.println("Login successfully");
> }
> }
> When I use IBM JDK and pass a relative path for the Kerberos keytab, it will
> throw error messages. According to the debug log, it always tries to read
> the keytab from the root path. See the debug logs below:
> 2017-04-19 02:29:13,982 DEBUG
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful
> kerberos logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,990 DEBUG
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos
> logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,991 DEBUG
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=,
> sampleName=Ops, always=false, type=DEFAULT, value=[GetGroups], valueName=Time)
> 2017-04-19 02:29:13,992 DEBUG
> [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and
> group related metrics
> [KRB_DBG_CFG] Config:main: Java config file:
> /opt/ibm/java/jre/lib/security/krb5.conf
> [KRB_DBG_CFG] Config:main: Loaded from Java config
> 2017-04-19 02:29:14,175 DEBUG [org.apache.hadoop.security.Groups] - Creating
> new Groups object
> 2017-04-19 02:29:14,178 DEBUG [org.apache.hadoop.util.NativeCodeLoader] -
> Trying to load the custom-built native-hadoop library...
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] -
> Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError:
> hadoop (Not found in java.library.path)
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] -
> java.library.path=/opt/ibm/java/jre/lib/amd64/compressedrefs:/opt/ibm/java/jre/lib/amd64:/usr/lib64:/usr/lib
> 2017-04-19 02:29:14,179 WARN [org.apache.hadoop.util.NativeCodeLoader] -
> Unable to load native-hadoop library for your platform... using builtin-java
> classes where applicable
> 2017-04-19 02:29:14,180 DEBUG
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling
> back to shell based
> 2017-04-19 02:29:14,180 DEBUG
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.util.Shell] - setsid exited
> with exit code 0
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.security.Groups] - Group
> mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> IBMJGSSProvider Build-Level: -20161128
> [JGSS_DBG_CRED] main JAAS config: principal=job/analytics
> [JGSS_DBG_CRED] main JAAS config: credsType=initiate and accept
> [JGSS_DBG_CRED] main config: useDefaultCcache=false
> [JGSS_DBG_CRED] main config: useCcache=null
> [JGSS_DBG_CRED] main config: useDefaultKeytab=false
> [JGSS_DBG_CRED] main config: useKeytab=//job.keytab
> [JGSS_DBG_CRED] main JAAS config: forwardable=false (default)
> [JGSS_DBG_CRED] main JAAS config: renewable=false (default)
> [JGSS_DBG_CRED] main JAAS config: proxiable=false (default)
> [JGSS_DBG_CRED] main JAAS config: tryFirstPass=false (default)
> [JGSS_DBG_CRED] main JAAS config: useFirstPass=false (default)
> [JGSS_DBG_CRED] main JAAS config: moduleBanner=false (default)
> [JGSS_DBG_CRED] main JAAS config: interactive login? no
> [JGSS_DBG_CRED] main JAAS config: refreshKrb5Config = true
> [KRB_DBG_CFG] Config:main: Java config file:
> /opt/ibm/java/jre/lib/security/krb5.conf
> [KRB_DBG_CFG] Config:main: Loaded from Java config
> [KRB_DBG_KDC] KdcComm:main: >>> KdcAccessibility: reset
> [KRB_DBG_KDC] KdcComm:main: >>> KdcAccessibility: reset
> [JGSS_DBG_CRED] main Try keytab for principal=job/analytics
> [KRB_DBG_KTAB] KeyTab:main: >>> KeyTab: trying to load keytab file
> /job.keytab
> [KRB_DBG_KTAB] KeyTab:main: >>> KeyTab: exception /job.keytab (No such file
> or directory)
> Key for the principal job/[email protected] not available in
> //job.keytab
> [KRB_DBG_CCHE] Credentials:main: >>> Credentials: Created Credentials with
> 0 keys. Key types:
> [JGSS_DBG_CRED] main Done retrieving Kerberos creds from keytab
> [JGSS_DBG_CRED] main Retrieving Kerberos creds from cache for
> principal=job/analytics
> [JGSS_DBG_CRED] main Non-interactive login; no callbacks necessary.
> [JGSS_DBG_CRED] main No Kerberos creds in cache for principal job/analytics
> [JGSS_DBG_CRED] main Doing Kerberos login for principal
> job/[email protected]
> 2017-04-19 02:29:14,381 DEBUG
> [org.apache.hadoop.security.UserGroupInformation] - hadoop login
> Exception in thread "main" java.io.IOException: Login failure for
> job/analytics from keytab job.keytab
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1103)
> at com.TestKrb.main(TestKrb.java:10)
> Caused by: javax.security.auth.login.FailedLoginException: Null key
> at
> com.ibm.security.jgss.i18n.I18NException.throwFailedLoginException(I18NException.java:1)
> at
> com.ibm.security.auth.module.Krb5LoginModule.a(Krb5LoginModule.java:355)
> at
> com.ibm.security.auth.module.Krb5LoginModule.b(Krb5LoginModule.java:515)
> at
> com.ibm.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:411)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
> at java.lang.reflect.Method.invoke(Method.java:508)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:788)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:196)
> at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)
> at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)
> at
> java.security.AccessController.doPrivileged(AccessController.java:686)
> at
> javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:719)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:593)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1092)
> ... 1 more
> In above log, the useKeytab=<value> entry is showing a <value> prefaced by a
> leading "//". It appears that HADOOP is adjusting the user supplied keytab
> file and most likely prefacing it with something like "FILE://", which would
> cause the resulting IBM normalized value to then be prefaced by "//" before
> the user supplied keytab file. This is the cause for why relative paths used
> with HADOOP are not working with IBM JVM's.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
