> On Feb. 10, 2017, 11:37 p.m., Greg Senia wrote:
> > Ship It!
While using doAS this does not work...
gint), stock_price_adj_close (type:
float)"}}}}]}},"DagId:":"hive_20170210183719_81144261-1a2b-4159-9e62-dc7bad4ebfc7:1","DagName:":""}},"Stage-2":{"Dependency
Collection":{}},"Stage-0":{"Move Operator":{"files:":{"hdfs
directory:":"true","destination:":"hdfs://tech/apps/hive/warehouse/gss_test_gss_test"}}}}},
endTime=Fri Feb 10 18:37:40 EST 2017}}]] after 3 retries. Quitting
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at
org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335)
at
org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188)
at
org.apache.atlas.kafka.KafkaNotification.createProducer(KafkaNotification.java:311)
at
org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:220)
at
org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:84)
at
org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:134)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:119)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:172)
at org.apache.atlas.hive.hook.HiveHook.access$300(HiveHook.java:85)
at org.apache.atlas.hive.hook.HiveHook$3.run(HiveHook.java:224)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1740)
at
org.apache.atlas.hive.hook.HiveHook.notifyAsPrivilegedAction(HiveHook.java:233)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:206)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.KafkaException:
javax.security.auth.login.LoginException: Could not login: the client is being
asked for a password, but the Kafka client code does not currently support
obtaining a password from the user. not available to garner authentication
information from the user
at
org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
at
org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:71)
at
org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
at
org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277)
... 19 more
Caused by: javax.security.auth.login.LoginException: Could not login: the
client is being asked for a password, but the Kafka client code does not
currently support obtaining a password from the user. not available to garner
authentication information from the user
at
com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940)
at
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
at
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at
org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:69)
at
org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:110)
at
org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:46)
at
org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68)
at
org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
- Greg
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56543/#review165205
-----------------------------------------------------------
On Feb. 10, 2017, 2:26 p.m., Nixon Rodrigues wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56543/
> -----------------------------------------------------------
>
> (Updated Feb. 10, 2017, 2:26 p.m.)
>
>
> Review request for atlas, keval bhatt, Madhan Neethiraj, Suma Shivaprasad,
> and Vimal Sharma.
>
>
> Bugs: ATLAS-1546
> https://issues.apache.org/jira/browse/ATLAS-1546
>
>
> Repository: atlas
>
>
> Description
> -------
>
> In a kerberized environment, Atlas hook uses JAAS configuration section named
> "KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment
> this configuration section is set to use the keytab and principal of
> HiveServer2 process. The hook running in HiveCLI might fail to authenticate
> with Kafka if the user can't read the configured keytab.
> Given that HiveCLI users would have performed kinit, the hook in HiveCLI
> should use the ticket-cache generated by kinit. When ticket cache is not
> available (for example in HiveServer2), the hook should use the configuration
> provided in KafkaClient JAAS section.
>
>
> Atlas Jaas properties
>
> atlas.jaas.ticketBased-KafkaClient.loginModuleControlFlag=required
> atlas.jaas.ticketBased-KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
> atlas.jaas.ticketBased-KafkaClient.option.useTicketCache=true
>
>
> Diffs
> -----
>
>
> common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java
> ff80eca
>
> common/src/test/java/org/apache/atlas/security/InMemoryJAASConfigurationTicketBasedKafkaClientTest.java
> PRE-CREATION
> common/src/test/resources/atlas-jaas.properties 90a5682
> notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 0534910
>
> Diff: https://reviews.apache.org/r/56543/diff/
>
>
> Testing
> -------
>
> Maven build completed without issue & executed mvn clean install and all the
> testcases are passing except few.
> Added a new unit testcase for TicketBasedKafkaClient.
>
> Deployed the new jars ( atlas-common & atlas-notification
> in??/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/ and
> tested hive hook on secure and simple env for HiveCli,Beeline clients.
> Entities on Atlas are created for tables created in Hive.
>
>
> Thanks,
>
> Nixon Rodrigues
>
>
