Repository: incubator-zeppelin Updated Branches: refs/heads/master aff2755eb -> 78c7b5567
Make zeppelin work with CDH5.7.0 ### What is this PR for? The downloadable zeppelin install wasn;t working with CDH 5.7.0. I had to change a few things to make it work nicely together. ### What type of PR is it? Bug Fix ### Todos ### What is the Jira issue? https://www.mail-archive.com/userszeppelin.incubator.apache.org/msg03471.html ### How should this be tested? Install local cdh5.7.0 (see https://github.com/krisgeus/ansible_local_cdh_hadoop) build zeppeling (with this patch) with the following options: mvn clean package -DskipTests -Pspark-1.6 -Phadoop-2.6 -Dspark.version=1.6.0-cdh5.7.0 -Dhadoop.version=2.6.0-cdh5.7.0 -Pvendor-repo ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? Didn't verify * Does this needs documentation? Don't think so Author: Kris Geusebroek <kgeusebroek@KgMBP2015.local> Closes #868 from krisgeus/master and squashes the following commits: e33d520 [Kris Geusebroek] Revert "Don't use extra driver classpath option since I use SPARK_CLASSPATH in spark-env.sh and using both is not supported" 488cce6 [Kris Geusebroek] Added logging and comments to clarify reason not throwing exception 146b524 [Kris Geusebroek] Don't use extra driver classpath option since I use SPARK_CLASSPATH in spark-env.sh and using both is not supported 24ea584 [Kris Geusebroek] method classServerUri not available in cdh5.7.0 Spark version. Only set config if variable is filled 50717dd [Kris Geusebroek] Use slf4j instead of parquet bundled one since parquet doesn't bundle it anymore Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/78c7b556 Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/78c7b556 Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/78c7b556 Branch: refs/heads/master Commit: 78c7b5567e7fb4985cecf147c39033c554dfc208 Parents: aff2755 Author: Kris Geusebroek <kgeusebroek@KgMBP2015.local> Authored: Fri May 13 08:38:11 2016 +0200 Committer: Felix Cheung <felixche...@apache.org> Committed: Wed May 25 21:04:49 2016 -0700 ---------------------------------------------------------------------- .../org/apache/zeppelin/spark/SparkInterpreter.java | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/78c7b556/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java ---------------------------------------------------------------------- diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java index 7d134ee..60613d3 100644 --- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java +++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java @@ -276,15 +276,21 @@ public class SparkInterpreter extends Interpreter { classServerUri = (String) classServer.invoke(interpreter.intp()); } catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException e) { - throw new InterpreterException(e); + // continue instead of: throw new InterpreterException(e); + // Newer Spark versions (like the patched CDH5.7.0 one) don't contain this method + logger.warn(String.format("Spark method classServerUri not available due to: [%s]", + e.getMessage())); } } SparkConf conf = new SparkConf() .setMaster(getProperty("master")) - .setAppName(getProperty("spark.app.name")) - .set("spark.repl.class.uri", classServerUri); + .setAppName(getProperty("spark.app.name")); + + if (classServerUri != null) { + conf.set("spark.repl.class.uri", classServerUri); + } if (jars.length > 0) { conf.setJars(jars);
