I could see "Ticket cache: KEYRING:persistent:1004:1004" in your env.
May be KEYRING persistent cache setting is causing trouble, Kerberos libraries to store the krb cache in a location and the Hadoop libraries can't seem to access it. Please refer these links, https://community.hortonworks.com/questions/818/ipa-kerberos-not-liking-my-kinit-ticket.html https://community.hortonworks.com/articles/11291/kerberos-cache-in-ipa-redhat-idm-keyring-solved-1.html Rakesh Intel On Wed, Sep 21, 2016 at 2:21 PM, kevin <[email protected]> wrote: > [hadoop@dmp1 ~]$ hdfs dfs -ls / > 16/09/20 15:00:44 WARN ipc.Client: Exception encountered while connecting > to the server : javax.security.sasl.SaslException: GSS initiate failed > [Caused by GSSException: No valid credentials provided (Mechanism level: > Failed to find any Kerberos tgt)] > ls: Failed on local exception: java.io.IOException: > javax.security.sasl.SaslException: > GSS initiate failed [Caused by GSSException: No valid credentials provided > (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local > host is: "dmp1.youedata.com/192.168.249.129"; destination host is: " > dmp1.youedata.com":9000; > [hadoop@dmp1 ~]$ klist > Ticket cache: KEYRING:persistent:1004:1004 > Default principal: [email protected] > > Valid starting Expires Service principal > 09/20/2016 14:57:34 09/21/2016 14:57:31 krbtgt/[email protected] > renew until 09/27/2016 14:57:31 > [hadoop@dmp1 ~]$ > > I have run kinit [email protected] before . > > 2016-09-21 10:14 GMT+08:00 Wei-Chiu Chuang <[email protected]>: > >> You need to run kinit command to authenticate before running hdfs dfs -ls >> command. >> >> Wei-Chiu Chuang >> >> On Sep 20, 2016, at 6:59 PM, kevin <[email protected]> wrote: >> >> Thank you Brahma Reddy Battula. >> It's because of my problerm of the hdfs-site config file and https >> ca configuration. >> now I can startup namenode and I can see the datanodes from the web. >> but When I try hdfs dfs -ls /: >> >> *[hadoop@dmp1 hadoop-2.7.3]$ hdfs dfs -ls /* >> *16/09/20 07:56:48 WARN ipc.Client: Exception encountered while >> connecting to the server : javax.security.sasl.SaslException: GSS initiate >> failed [Caused by GSSException: No valid credentials provided (Mechanism >> level: Failed to find any Kerberos tgt)]* >> *ls: Failed on local exception: java.io.IOException: >> javax.security.sasl.SaslException: GSS initiate failed [Caused by >> GSSException: No valid credentials provided (Mechanism level: Failed to >> find any Kerberos tgt)]; Host Details : local host is: >> "dmp1.example.com/192.168.249.129 >> <http://dmp1.example.com/192.168.249.129>"; destination host is: "dmp1.* >> *example**.com":9000; * >> >> current user is hadoop which startup hdfs , and I have add addprinc >> hadoop with commond : >> kadmin.local -q "addprinc hadoop" >> >> >> 2016-09-20 17:33 GMT+08:00 Brahma Reddy Battula < >> [email protected]>: >> >>> Seems to be property problem.. it should be *principal* ( “l” is >>> missed). >>> >>> >>> >>> *<property>* >>> >>> * <name>dfs.secondary.namenode.kerberos.principa</name>* >>> >>> * <value>hadoop/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> >>> >>> >>> >>> For namenode httpserver start fail, please check rakesh comments.. >>> >>> >>> >>> This is probably due to some missing configuration. >>> >>> Could you please re-check the ssl-server.xml, keystore and truststore >>> properties: >>> >>> >>> >>> ssl.server.keystore.location >>> >>> ssl.server.keystore.keypassword >>> >>> ssl.client.truststore.location >>> >>> ssl.client.truststore.password >>> >>> >>> >>> >>> >>> --Brahma Reddy Battula >>> >>> >>> >>> *From:* kevin [mailto:[email protected]] >>> *Sent:* 20 September 2016 16:53 >>> *To:* Rakesh Radhakrishnan >>> *Cc:* user.hadoop >>> *Subject:* Re: hdfs2.7.3 kerberos can not startup >>> >>> >>> >>> thanks, but my issue is name node could *Login successful,but second >>> namenode couldn't. and name node got a HttpServer.start() threw a non Bind >>> IOException:* >>> >>> >>> >>> hdfs-site.xml: >>> >>> >>> >>> *<property>* >>> >>> * <name>dfs.webhdfs.enabled</name>* >>> >>> * <value>true</value>* >>> >>> *</property>* >>> >>> >>> >>> *<property>* >>> >>> * <name>dfs.block.access.token.enable</name>* >>> >>> * <value>true</value>* >>> >>> *</property>* >>> >>> >>> >>> *<!-- NameNode security config -->* >>> >>> *<property>* >>> >>> * <name>dfs.namenode.kerberos.pr >>> <http://dfs.namenode.kerberos.pr>incipal</name>* >>> >>> * <value>hadoop/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.namenode.keytab.file</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.https.port</name>* >>> >>> * <value>50470</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.namenode.https-address</name>* >>> >>> * <value>dmp1.example.com:50470 >>> <http://dmp1.example.com:50470/></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.namenode.kerberos.in >>> <http://dfs.namenode.kerberos.in>ternal.spnego.principa</name>* >>> >>> * <value>HTTP/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.web.authentication.kerberos.keytab</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.http.policy</name>* >>> >>> * <value>HTTPS_ONLY</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.https.enable</name>* >>> >>> * <value>true</value>* >>> >>> *</property>* >>> >>> >>> >>> >>> >>> *<!-- secondary NameNode security config -->* >>> >>> *<property>* >>> >>> * <name>dfs.namenode.secondary.http-address</name>* >>> >>> * <value>dmp1.example.com:50090 >>> <http://dmp1.example.com:50090/></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.secondary.namenode.keytab.file</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.secondary.namenode.kerberos.principa</name>* >>> >>> * <value>hadoop/[email protected] <[email protected]></value>* >>> >>> *</property> * >>> >>> *<property>* >>> >>> * >>> <name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>* >>> >>> * <value>HTTP/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.namenode.secondary.https-port</name>* >>> >>> * <value>50470</value>* >>> >>> *</property>* >>> >>> >>> >>> >>> >>> *<!-- JournalNode security config -->* >>> >>> >>> >>> *<property>* >>> >>> * <name>dfs.journalnode.keytab.file</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.journalnode.kerberos.principa</name>* >>> >>> * <value>hadoop/[email protected] <[email protected]></value>* >>> >>> *</property> * >>> >>> *<property>* >>> >>> * <name>dfs.journalnode.kerberos.internal.spnego.principa</name>* >>> >>> * <value>HTTP/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.web.authentication.kerberos.keytab</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> >>> >>> >>> >>> *<!-- DataNode security config -->* >>> >>> *<property>* >>> >>> * <name>dfs.datanode.kerberos.pr >>> <http://dfs.datanode.kerberos.pr>incipal</name>* >>> >>> * <value>hadoop/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.datanode.keytab.file</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.datanode.data.dir.pe >>> <http://dfs.datanode.data.dir.pe>rm</name>* >>> >>> * <value>700</value>* >>> >>> *</property>* >>> >>> >>> >>> *<!-- datanode SASL-->* >>> >>> *<property>* >>> >>> * <name>dfs.datanode.address</name>* >>> >>> * <value>0.0.0.0:61004 <http://0.0.0.0:61004/></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.datanode.http.address</name>* >>> >>> * <value>0.0.0.0:61006 <http://0.0.0.0:61006/></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.datanode.https.address</name>* >>> >>> * <value>0.0.0.0:50470 <http://0.0.0.0:50470/></value>* >>> >>> *</property>* >>> >>> >>> >>> *<property>* >>> >>> * <name>dfs.data.transfer.protection</name>* >>> >>> * <value>integrity</value>* >>> >>> *</property>* >>> >>> >>> >>> *<property>* >>> >>> * <name>dfs.web.authentication.kerberos.principal</name>* >>> >>> * <value>HTTP/[email protected] <[email protected]></value>* >>> >>> *</property>* >>> >>> *<property>* >>> >>> * <name>dfs.web.authentication.kerberos.keytab</name>* >>> >>> * <value>/etc/hadoop/conf/hdfs.keytab</value>* >>> >>> *</property>* >>> >>> >>> >>> *and [hadoop@dmp1 hadoop-2.7.3]$ klist -ket /etc/hadoop/conf/hdfs.keytab* >>> >>> >>> >>> >>> >>> *Keytab name: FILE:/etc/hadoop/conf/hdfs.keytab* >>> >>> *KVNO Timestamp Principal* >>> >>> *---- ------------------- >>> ------------------------------------------------------* >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 16:00:41 hdfs/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 16:00:41 HTTP/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (aes256-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (aes128-cts-hmac-sha1-96) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (des3-cbc-sha1) * >>> >>> * 2 09/19/2016 20:21:03 hadoop/[email protected] >>> <[email protected]> (arcfour-hmac) * >>> >>> >>> >>> 2016-09-20 15:52 GMT+08:00 Rakesh Radhakrishnan <[email protected]>: >>> >>> >>>>>>Caused by: javax.security.auth.login.LoginException: Unable to >>> obtain password from user >>> >>> >>> >>> Could you please check kerberos principal name is specified correctly in >>> >>> "hdfs-site.xml", which is used to authenticate against Kerberos. >>> >>> >>> >>> If keytab file defined in "hdfs-site.xml" and doesn't exists or wrong >>> path, you will see >>> >>> this error. So, please verify the path and the keytab filename correctly >>> >>> configured. >>> >>> >>> >>> I hope hadoop discussion thread, https://goo.gl/M6l3vv may help you. >>> >>> >>> >>> >>> >>> >>>>>>>2016-09-20 00:54:06,665 INFO org.apache.hadoop.http.HttpServer2: >>> HttpServer.start() threw a non Bind IOException >>> >>> java.io.IOException: !JsseListener: java.lang.NullPointerException >>> >>> >>> >>> This is probably due to some missing configuration. >>> >>> Could you please re-check the ssl-server.xml, keystore and truststore >>> properties: >>> >>> >>> >>> ssl.server.keystore.location >>> >>> ssl.server.keystore.keypassword >>> >>> ssl.client.truststore.location >>> >>> ssl.client.truststore.password >>> >>> >>> >>> Rakesh >>> >>> >>> >>> On Tue, Sep 20, 2016 at 10:53 AM, kevin <[email protected]> wrote: >>> >>> *hi,all:* >>> >>> *My environment : Centos7.2 hadoop2.7.3 jdk1.8* >>> >>> *after I config hdfs with kerberos ,I can't start up with >>> sbin/start-dfs.sh* >>> >>> >>> >>> *::namenode log as below * >>> >>> >>> >>> *STARTUP_MSG: build = Unknown -r Unknown; compiled by 'root' on >>> 2016-09-18T09:05Z* >>> >>> *STARTUP_MSG: java = 1.8.0_102* >>> >>> *************************************************************/* >>> >>> *2016-09-20 00:54:05,822 INFO >>> org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal >>> handlers for [TERM, HUP, INT]* >>> >>> *2016-09-20 00:54:05,825 INFO >>> org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode []* >>> >>> *2016-09-20 00:54:06,078 INFO >>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >>> hadoop-metrics2.properties* >>> >>> *2016-09-20 00:54:06,149 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >>> period at 10 second(s).* >>> >>> *2016-09-20 00:54:06,149 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system >>> started* >>> >>> *2016-09-20 00:54:06,151 INFO >>> org.apache.hadoop.hdfs.server.namenode.NameNode: fs.defaultFS is >>> hdfs://dmp1.example.com:9000 <http://dmp1.example.com:9000/>* >>> >>> *2016-09-20 00:54:06,152 INFO >>> org.apache.hadoop.hdfs.server.namenode.NameNode: Clients are to use >>> dmp1.example.com:9000 <http://dmp1.example.com:9000/> to access this >>> namenode/service.* >>> >>> *2016-09-20 00:54:06,446 INFO >>> org.apache.hadoop.security.UserGroupInformation: Login successful for user >>> hadoop/[email protected] <[email protected]> using >>> keytab file /etc/hadoop/conf/hdfs.keytab* >>> >>> *2016-09-20 00:54:06,472 INFO org.apache.hadoop.hdfs.DFSUtil: Starting >>> web server as: HTTP/[email protected] >>> <[email protected]>* >>> >>> *2016-09-20 00:54:06,475 INFO org.apache.hadoop.hdfs.DFSUtil: Starting >>> Web-server for hdfs at: https://dmp1.example.com:50470 >>> <https://dmp1.example.com:50470/>* >>> >>> *2016-09-20 00:54:06,517 INFO org.mortbay.log: Logging to >>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >>> org.mortbay.log.Slf4jLog* >>> >>> *2016-09-20 00:54:06,533 INFO >>> org.apache.hadoop.security.authentication.server.AuthenticationFilter: >>> Unable to initialize FileSignerSecretProvider, falling back to use random >>> secrets.* >>> >>> *2016-09-20 00:54:06,542 INFO org.apache.hadoop.http.HttpRequestLog: >>> Http request log for http.requests.namenode is not defined* >>> >>> *2016-09-20 00:54:06,546 INFO org.apache.hadoop.http.HttpServer2: Added >>> global filter 'safety' >>> (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)* >>> >>> *2016-09-20 00:54:06,548 INFO org.apache.hadoop.http.HttpServer2: Added >>> filter static_user_filter >>> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to >>> context hdfs* >>> >>> *2016-09-20 00:54:06,548 INFO org.apache.hadoop.http.HttpServer2: Added >>> filter static_user_filter >>> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to >>> context static* >>> >>> *2016-09-20 00:54:06,548 INFO org.apache.hadoop.http.HttpServer2: Added >>> filter static_user_filter >>> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to >>> context logs* >>> >>> *2016-09-20 00:54:06,653 INFO org.apache.hadoop.http.HttpServer2: Added >>> filter 'org.apache.hadoop.hdfs.web.Au >>> <http://org.apache.hadoop.hdfs.web.au/>thFilter' >>> (class=org.apache.hadoop.hdfs.web.AuthFilter)* >>> >>> *2016-09-20 00:54:06,654 INFO org.apache.hadoop.http.HttpServer2: >>> addJerseyResourcePackage: >>> packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, >>> pathSpec=/webhdfs/v1/** >>> >>> *2016-09-20 00:54:06,657 INFO org.apache.hadoop.http.HttpServer2: Adding >>> Kerberos (SPNEGO) filter to getDelegationToken* >>> >>> *2016-09-20 00:54:06,658 INFO org.apache.hadoop.http.HttpServer2: Adding >>> Kerberos (SPNEGO) filter to renewDelegationToken* >>> >>> *2016-09-20 00:54:06,658 INFO org.apache.hadoop.http.HttpServer2: Adding >>> Kerberos (SPNEGO) filter to cancelDelegationToken* >>> >>> *2016-09-20 00:54:06,659 INFO org.apache.hadoop.http.HttpServer2: Adding >>> Kerberos (SPNEGO) filter to fsck* >>> >>> *2016-09-20 00:54:06,659 INFO org.apache.hadoop.http.HttpServer2: Adding >>> Kerberos (SPNEGO) filter to imagetransfer* >>> >>> *2016-09-20 00:54:06,665 WARN org.mortbay.log: >>> java.lang.NullPointerException* >>> >>> *2016-09-20 00:54:06,665 INFO org.apache.hadoop.http.HttpServer2: >>> HttpServer.start() threw a non Bind IOException* >>> >>> *java.io.IOException: !JsseListener: java.lang.NullPointerException* >>> >>> *at >>> org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)* >>> >>> *at >>> org.apache.hadoop.security.ssl.SslSocketConnectorSecure.newServerSocket(SslSocketConnectorSecure.java:47)* >>> >>> *at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)* >>> >>> *at >>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:914)* >>> >>> *at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:856)* >>> >>> *at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.st >>> <http://namenode.namenodehttpserver.st/>art(NameNodeHttpServer.java:142)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:753)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:639)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:812)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:796)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)* >>> >>> >>> >>> >>> >>> *::second namenode log as below * >>> >>> >>> >>> *STARTUP_MSG: build = Unknown -r Unknown; compiled by 'root' on >>> 2016-09-18T09:05Z* >>> >>> *STARTUP_MSG: java = 1.8.0_102* >>> >>> *************************************************************/* >>> >>> *2016-09-20 00:54:14,885 INFO >>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: registered UNIX >>> signal handlers for [TERM, HUP, INT]* >>> >>> *2016-09-20 00:54:15,263 FATAL >>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Failed to start >>> secondary namenode* >>> >>> *java.io.IOException: Login failure for hadoop from keytab >>> /etc/hadoop/conf/hdfs.keytab: javax.security.auth.login.LoginException: >>> Unable to obtain password from user* >>> >>> >>> >>> *at >>> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:963)* >>> >>> *at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:246)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:217)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:192)* >>> >>> *at >>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:671)* >>> >>> *Caused by: javax.security.auth.login.LoginException: Unable to obtain >>> password from user* >>> >>> >>> >>> *at >>> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)* >>> >>> *at >>> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)* >>> >>> *at >>> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)* >>> >>> *at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* >>> >>> *at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)* >>> >>> *at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* >>> >>> *at java.lang.reflect.Method.invoke(Method.java:498)* >>> >>> *at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)* >>> >>> *at >>> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)* >>> >>> *at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)* >>> >>> *at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)* >>> >>> *at java.security.AccessController.doPrivileged(Native Method)* >>> >>> *at >>> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)* >>> >>> *at javax.security.auth.login.LoginContext.login(LoginContext.java:587)* >>> >>> *at >>> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:954)* >>> >>> *... 4 more* >>> >>> >>> >>> >>> >> >> >> >
