[
https://issues.apache.org/jira/browse/GEARPUMP-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210582#comment-16210582
]
ASF GitHub Bot commented on GEARPUMP-355:
-----------------------------------------
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/231#discussion_r145601399
--- Diff:
experiments/yarn/src/main/scala/org/apache/gearpump/experiments/yarn/client/AppMasterResolver.scala
---
@@ -75,3 +64,38 @@ class AppMasterResolver(yarnClient: YarnClient, system:
ActorSystem) {
result
}
}
+
+object AppMasterResolver {
+
+ val LOG = LogUtil.getLogger(getClass)
+
+ def resolveAppMasterAddress(report: ApplicationReport, system:
ActorSystem): ActorRef = {
+ val appMasterPath = s"${report.getOriginalTrackingUrl}" +
"supervisor-actor-path"
+ LOG.info(s"appMasterPath=$appMasterPath")
+
+ val connectionFactory: URLConnectionFactory = URLConnectionFactory
+ .newDefaultURLConnectionFactory(new YarnConfiguration())
+ val url: URL = new URL(appMasterPath)
+ val connection: HttpURLConnection =
connectionFactory.openConnection(url)
+ .asInstanceOf[HttpURLConnection]
+ connection.setInstanceFollowRedirects(true)
+ connection.connect()
+ val status = connection.getResponseCode
+
+ var stream: java.io.InputStream = connection.getErrorStream
+ if(stream == null) {
+ stream = connection.getInputStream
+ }
+ if(status == 200) {
--- End diff --
fix style: need a space before `(`
> AppMasterResolver fails to run against a kerberized Hadoop cluster
> ------------------------------------------------------------------
>
> Key: GEARPUMP-355
> URL: https://issues.apache.org/jira/browse/GEARPUMP-355
> Project: Apache Gearpump
> Issue Type: Bug
> Components: security, yarn
> Affects Versions: 0.8.4
> Reporter: Timea Magyar
> Fix For: 0.8.4
>
>
> When trying to launch a Gearpump cluster in a kerberized Hadoop/Yarn
> environment, after the Application Master address has been resolved as a
> prerequisite, the YarnAppMaster (responsible for starting GearPump masters,
> workers, UI servers as Yarn containers) address (actor reference) must be
> obtained via Kerberos/Spnego. (Kerberos over http)
> The current implementation for this resides in the AppMasterResolver class
> and is using an apache http client (version 3.x) for establishing a
> connection to the Application Master and obtain the above YarnAppMaster actor
> reference. Since the apache http client does not support the negotiate
> authentication scheme in version 3.x (required for a connection over
> kerberos/spnego) this step will always fail in a kerberized Yarn/Hadoop
> cluster set-up.
> I tested this in a secured/kerberized CDH 5.7.5 environment. I would like to
> provide a patch for this by adapting the SPNEGO-enabled Hadoop web
> connection code from WebHDFS.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
