Repository: zeppelin Updated Branches: refs/heads/master 62aec9fbd -> 08a981403
[Zeppelin 2367] Hive JDBC proxy user option should be available even without kerberos ### What is this PR for? Hive JDBC proxy user option should be available generically. ### What type of PR is it? [Improvement] ### What is the Jira issue? * [Zeppelin 2367](https://issues.apache.org/jira/browse/ZEPPELIN-2367) ### How should this be tested? Enable Shiro authentication and set `zeppelin.jdbc.auth.type` as `SIMPLE` in the interpreter setting, and observe the connection string for the Hive. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need an update? N/A * Is there breaking changes for older versions? N/A * Does this needs documentation? Yes Author: Prabhjyot Singh <prabhjyotsi...@gmail.com> Closes #2229 from prabhjyotsingh/ZEPPELIN-2367 and squashes the following commits: 84b5e55b2 [Prabhjyot Singh] add logger.warn for hive and impersonation 45c90a8e2 [Prabhjyot Singh] improve doc 9fee9d2a9 [Prabhjyot Singh] replace hive with generic method a348e969a [Prabhjyot Singh] revert "zeppelin.jdbc.auth.kerberos.proxy.enable" behaviour e2bdbb2ad [Prabhjyot Singh] include e as inner exception c180f5ce2 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367 1802b453f [Prabhjyot Singh] remove hive string from logger 513987a28 [Prabhjyot Singh] apply genric logic to appendProxyUserToURL 3fa2b1e98 [Prabhjyot Singh] change name to appendProxyUserToURL a75167415 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367 4c382eefa [Prabhjyot Singh] log user details as well d51e770b2 [Prabhjyot Singh] add doc in jdbc.md 01b18b9d2 [Prabhjyot Singh] add doc (reverted from commit ee8a6b524c481210486761032cb1f5fd6266bb54) 40489c89d [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367 ee8a6b524 [Prabhjyot Singh] add doc 8999d93ae [Prabhjyot Singh] ZEPPELIN-2367: Hive JDBC proxy user option should be avail even without kerberos Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/08a98140 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/08a98140 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/08a98140 Branch: refs/heads/master Commit: 08a9814033e216eb2d340c49d171f3b66c19eb61 Parents: 62aec9f Author: Prabhjyot Singh <prabhjyotsi...@gmail.com> Authored: Sun Apr 30 00:15:19 2017 +0530 Committer: Prabhjyot Singh <prabhjyotsi...@gmail.com> Committed: Wed May 3 21:17:16 2017 +0530 ---------------------------------------------------------------------- docs/interpreter/jdbc.md | 35 ++++++++--- .../apache/zeppelin/jdbc/JDBCInterpreter.java | 66 +++++++++++--------- .../jdbc/security/JDBCSecurityImpl.java | 5 +- .../interpreter/InterpreterException.java | 4 ++ 4 files changed, 68 insertions(+), 42 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/docs/interpreter/jdbc.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/jdbc.md b/docs/interpreter/jdbc.md index f66ad6d..f10444b 100644 --- a/docs/interpreter/jdbc.md +++ b/docs/interpreter/jdbc.md @@ -171,7 +171,7 @@ There are more JDBC interpreter properties you can specify like below. </tr> <tr> <td>zeppelin.jdbc.auth.kerberos.proxy.enable</td> - Â Â Â <td>When auth type is Kerberos, enable/disable Kerberos proxy with the login user to get the connection. Default value is true.</td> + <td>When auth type is Kerberos, enable/disable Kerberos proxy with the login user to get the connection. Default value is true.</td> </tr> <tr> <td>default.jceks.file</td> @@ -202,7 +202,7 @@ To bind the interpreters created in the interpreter setting page, click the gear <img src="../assets/themes/zeppelin/img/docs-img/click_interpreter_binding_button.png" width="600px" /> -Select(blue) or deselect(white) the interpreter buttons depending on your use cases. +Select(blue) or deselect(white) the interpreter buttons depending on your use cases. If you need to use more than one interpreter in the notebook, activate several buttons. Don't forget to click `Save` button, or you will face `Interpreter *** is not found` error. @@ -285,7 +285,7 @@ An example settings of interpreter for the two data sources, each of which has i </table> ##### Usage -Test of execution *precode* for each data source. +Test of execution *precode* for each data source. ```sql %jdbc @@ -480,7 +480,7 @@ Here are some examples you can refer to. Including the below connectors, you can [Maven Repository: com.amazonaws:aws-java-sdk-redshift](https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-redshift) -### Apache Hive +### Apache Hive <img src="../assets/themes/zeppelin/img/docs-img/hive_setting.png" width="600px" /> @@ -507,12 +507,11 @@ Here are some examples you can refer to. Including the below connectors, you can <td>hive_password</td> </tr> <tr> - <td>hive.proxy.user</td> - <td>true or false</td> + <td>default.proxy.user.property</td> + <td>Example value: hive.server2.proxy.user</td> + </tr> </table> -Connection to Hive JDBC with a proxy user can be disabled with `hive.proxy.user` property (set to true by default) - [Apache Hive 1 JDBC Driver Docs](https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC) [Apache Hive 2 JDBC Driver Docs](https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC) @@ -534,6 +533,26 @@ Connection to Hive JDBC with a proxy user can be disabled with `hive.proxy.user` [Maven Repository : org.apache.hive:hive-jdbc](https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc) +##### Impersonation +When Zeppelin server is running with authentication enabled, then the interpreter can utilize Hive's user proxy feature i.e. send extra parameter for creating and running a session ("hive.server2.proxy.user=": "${loggedInUser}"). This is particularly useful when multiple users are sharing a notebooks. + +To enable this set following: + - `zeppelin.jdbc.auth.type` as `SIMPLE` or `KERBEROS` (if required) in the interpreter setting. + - `${prefix}.proxy.user.property` as `hive.server2.proxy.user` + Example configuration + + *Properties* + + | name | value | + |:------------------------- |:--------------------------------------------------------------------------------------------------| + | hive.driver | org.apache.hive.jdbc.HiveDriver | + | hive.password | | + | hive.url | jdbc:hive2://hive-server-host:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 | + | hive.proxy.user.property | hive.proxy.user.property | + | zeppelin.jdbc.auth.type | SIMPLE | + + + ### Apache Phoenix Phoenix supports `thick` and `thin` connection types: http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java ---------------------------------------------------------------------- diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java index 95b2a1a..3272bc1 100644 --- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java +++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java @@ -26,7 +26,6 @@ import java.util.ArrayList; import java.util.HashMap; import java.util.HashSet; import java.util.List; -import java.util.Map; import java.util.Properties; import java.util.Set; @@ -179,10 +178,6 @@ public class JDBCInterpreter extends Interpreter { } logger.debug("JDBC PropretiesMap: {}", basePropretiesMap); - if (!isEmpty(property.getProperty("zeppelin.jdbc.auth.type"))) { - JDBCSecurityImpl.createSecureConfiguration(property); - } - setMaxLineResults(); } @@ -355,36 +350,25 @@ public class JDBCInterpreter extends Interpreter { } else { UserGroupInformation.AuthenticationMethod authType = JDBCSecurityImpl.getAuthtype(property); + final String connectionUrl = appendProxyUserToURL(url, user, propertyKey); + + JDBCSecurityImpl.createSecureConfiguration(property, authType); switch (authType) { case KERBEROS: if (user == null || "false".equalsIgnoreCase( - property.getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable"))) { - connection = getConnectionFromPool(url, user, propertyKey, properties); + property.getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable"))) { + connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties); } else { - if (url.trim().startsWith("jdbc:hive")) { - StringBuilder connectionUrl = new StringBuilder(url); - Integer lastIndexOfUrl = connectionUrl.indexOf("?"); - if (lastIndexOfUrl == -1) { - lastIndexOfUrl = connectionUrl.length(); - } - boolean hasProxyUser = property.containsKey("hive.proxy.user"); - if (!hasProxyUser || !property.getProperty("hive.proxy.user").equals("false")){ - logger.debug("Using hive proxy user"); - connectionUrl.insert(lastIndexOfUrl, ";hive.server2.proxy.user=" + user + ";"); - } - connection = getConnectionFromPool(connectionUrl.toString(), - user, propertyKey, properties); + if (basePropretiesMap.get(propertyKey).containsKey("proxy.user.property")) { + connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties); } else { UserGroupInformation ugi = null; try { ugi = UserGroupInformation.createProxyUser( - user, UserGroupInformation.getCurrentUser()); + user, UserGroupInformation.getCurrentUser()); } catch (Exception e) { logger.error("Error in getCurrentUser", e); - StringBuilder stringBuilder = new StringBuilder(); - stringBuilder.append(e.getMessage()).append("\n"); - stringBuilder.append(e.getCause()); - throw new InterpreterException(stringBuilder.toString()); + throw new InterpreterException("Error in getCurrentUser", e); } final String poolKey = propertyKey; @@ -392,28 +376,48 @@ public class JDBCInterpreter extends Interpreter { connection = ugi.doAs(new PrivilegedExceptionAction<Connection>() { @Override public Connection run() throws Exception { - return getConnectionFromPool(url, user, poolKey, properties); + return getConnectionFromPool(connectionUrl, user, poolKey, properties); } }); } catch (Exception e) { logger.error("Error in doAs", e); - StringBuilder stringBuilder = new StringBuilder(); - stringBuilder.append(e.getMessage()).append("\n"); - stringBuilder.append(e.getCause()); - throw new InterpreterException(stringBuilder.toString()); + throw new InterpreterException("Error in doAs", e); } } } break; default: - connection = getConnectionFromPool(url, user, propertyKey, properties); + connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties); } } return connection; } + private String appendProxyUserToURL(String url, String user, String propertyKey) { + StringBuilder connectionUrl = new StringBuilder(url); + + if (user != null && !user.equals("anonymous") && + basePropretiesMap.get(propertyKey).containsKey("proxy.user.property")) { + + Integer lastIndexOfUrl = connectionUrl.indexOf("?"); + if (lastIndexOfUrl == -1) { + lastIndexOfUrl = connectionUrl.length(); + } + logger.info("Using proxy user as :" + user); + logger.info("Using proxy property for user as :" + + basePropretiesMap.get(propertyKey).getProperty("proxy.user.property")); + connectionUrl.insert(lastIndexOfUrl, ";" + + basePropretiesMap.get(propertyKey).getProperty("proxy.user.property") + "=" + user + ";"); + } else if (user != null && !user.equals("anonymous") && url.contains("hive")) { + logger.warn("User impersonation for hive has changed please refer: http://zeppelin.apache"; + + ".org/docs/latest/interpreter/jdbc.html#apache-hive"); + } + + return connectionUrl.toString(); + } + private String getPassword(Properties properties) throws IOException { if (isNotEmpty(properties.getProperty(PASSWORD_KEY))) { return properties.getProperty(PASSWORD_KEY); http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java ---------------------------------------------------------------------- diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java index 32a7990..25959e1 100644 --- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java +++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java @@ -38,9 +38,8 @@ public class JDBCSecurityImpl { /*** * @param properties */ - public static void createSecureConfiguration(Properties properties) { - AuthenticationMethod authType = getAuthtype(properties); - + public static void createSecureConfiguration(Properties properties, + AuthenticationMethod authType) { switch (authType) { case KERBEROS: Configuration conf = new http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java index 30c1c0a..ebd184e 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java @@ -31,4 +31,8 @@ public class InterpreterException extends RuntimeException { super(m); } + public InterpreterException(String msg, Throwable t) { + super(msg, t); + } + }
