Github user titikakatoo commented on the issue:
https://github.com/apache/incubator-gearpump/pull/231
@huafengw
I can provide you the following sanitized extract from the logs when trying
to use gearpump on a kerberized hadoop cluster without the patch:
> [HttpClient] Java version: 1.8.0_141
> [HttpClient] Java vendor: Oracle Corporation
> [HttpClient] Java class path: <classpath>
> [HttpClient] Operating system name: <>
> [HttpClient] Operating system architecture: <>
> [HttpClient] Operating system version: <>
> [HttpClient] SUN 1.8: <>
> [HttpClient] SunRsaSign 1.8: <>
> [HttpClient] SunEC 1.8: <>
> [HttpClient] SunJSSE 1.8: <>
> [HttpClient] SunJCE 1.8: <>
> [HttpClient] SunJGSS 1.8: Sun **(Kerberos v5, SPNEGO)**
> [HttpClient] SunSASL 1.8: <>
> [HttpClient] XMLDSig 1.8: <>
> [HttpClient] SunPCSC 1.8: <>
> [HttpClient] SaslPlainServer 1.0: SASL PLAIN Authentication Server
> [DefaultHttpParams] Set parameter http.useragent = **Jakarta
Commons-HttpClient/3.1**
> [DefaultHttpParams] Set parameter http.protocol.version = HTTP/1.1
> [DefaultHttpParams] Set parameter http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
> [DefaultHttpParams] Set parameter http.protocol.cookie-policy = default
> [DefaultHttpParams] Set parameter http.protocol.element-charset = US-ASCII
> [DefaultHttpParams] Set parameter http.protocol.content-charset =
ISO-8859-1
> [DefaultHttpParams] Set parameter http.method.retry-handler =
org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@54f5f647
> [DefaultHttpParams] Set parameter http.dateparser.patterns = <>
> [AppMasterResolver]
appMasterPath=https://<proxy_host>:<proxy_port>/proxy/<application_id>//supervisor-actor-path
> [HttpConnection] Open connection to <proxy_host>:<proxy_port>
> [header] >> "GET /proxy/<application_id>//supervisor-actor-path
HTTP/1.1[\r][\n]"
> [HttpMethodBase] Adding Host request header
> [header] >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
> [header] >> "Host: <proxy_host>:<proxy_port>[\r][\n]"
> [header] >> "[\r][\n]"
> [header] << "HTTP/1.1 401 Authentication required[\r][\n]"
> [header] << "HTTP/1.1 401 Authentication required[\r][\n]"
> [header] << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
> [header] << "Date: Thu, 12 Oct 2017 09:06:05 GMT[\r][\n]"
> [header] << "Pragma: no-cache[\r][\n]"
> [header] << "Date: Thu, 12 Oct 2017 09:06:05 GMT[\r][\n]"
> [header] << "Pragma: no-cache[\r][\n]"
> [header] << "Content-Type: text/html; charset=iso-8859-1[\r][\n]"
> [header] << "X-FRAME-OPTIONS: SAMEORIGIN[\r][\n]"
> [header] << "WWW-Authenticate: Negotiate[\r][\n]"
> [header] << "Set-Cookie: hadoop.auth=; Path=/; Secure; HttpOnly[\r][\n]"
> [header] << "Content-Length: 1452[\r][\n]"
> [header] << "Server: Jetty(6.1.26.cloudera.4)[\r][\n]"
> [header] << "[\r][\n]"
> [CookieSpec] Unrecognized cookie attribute: name=HttpOnly, value=null
> [HttpMethodBase] Cookie accepted: "$Version=0; hadoop.auth=; $Path=/"
> [HttpMethodDirector] Authorization required
> [AuthChallengeProcessor] Supported authentication schemes in the order of
preference: **[ntlm, digest, basic]**
> [AuthChallengeProcessor] Challenge for ntlm authentication scheme not
available
> [AuthChallengeProcessor] Challenge for digest authentication scheme not
available
> [AuthChallengeProcessor] Challenge for basic authentication scheme not
available
> [HttpMethodDirector] **Unable to respond to any of these challenges:
{negotiate=Negotiate}**
> [AppMasterResolver] Failed to connect YarnAppMaster(tried 1)... Fail to
resolve AppMaster address, please make sure
https://<proxy_host>:<proxy_port>/proxy/<application_id>/ is accessible...
Currently gearpump uses the apache http client version 3.1. This client
only supports the authentication schemes [ntlm, digest, basic] (see logs). As
you can further see in the logs: Unable to respond to any of these challenges:
{negotiate=Negotiate}, a communication via kerberos spnego is not possible in
this case.
After the patch trying to get the active configuration from YarnAppmaster
produces the following sanitized logs:
> [Client] IPC Client (<>) connection to <host><port> from kerberos
principal: starting, having connections 1
> [SaslRpcClient] reading next wrapped RPC packet
> [Client] IPC Client (<>) connection to <host><port> from kerberos
principal sending #0
> [SaslRpcClient] wrapping token of length:<length>
> [SaslRpcClient] unwrapping token of length:<length>
> [Client] IPC Client (<>) connection to <host><port> from kerberos
principal got value #0
> [ProtobufRpcEngine] Call: getApplicationReport took 224ms
> [AppMasterResolver$]
appMasterPath=https://<host>:<port>/proxy/application_<id>/supervisor-actor-path
> [FileBasedKeyStoresFactory] CLIENT TrustStore: <client_trust_store>
> [ReloadingX509TrustManager] Loaded truststore '<client_trust_store>'
> [FileBasedKeyStoresFactory] CLIENT Loaded TrustStore: <client_trust_store>
> [URLConnectionFactory] open URL connection
> [AppMasterResolver$] **Successfully resolved AppMaster address:
akka.tcp://GearpumpAM@<host>:<port>/user/appMaster**
> **ActiveConfig(Config<config>)**
> [RemoteActorRefProvider$RemotingTerminator] Shutting down remote daemon.
> [RemoteActorRefProvider$RemotingTerminator] Remote daemon shut down;
proceeding with flushing remote transports.
> [RemoteActorRefProvider$RemotingTerminator] Remoting shut down.
---
