This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 6d55732 [ZEPPELIN-4618] Zeppelin on Kubernetes does not automatically
configure Spark on Kubernetes
6d55732 is described below
commit 6d55732150bd36f2017ebc74896a4a5e09e3abc7
Author: Lee moon soo <m...@apache.org>
AuthorDate: Thu Feb 13 16:14:39 2020 -0800
[ZEPPELIN-4618] Zeppelin on Kubernetes does not automatically configure
Spark on Kubernetes
### What is this PR for?
Since https://github.com/apache/zeppelin/pull/3577, Zeppelin on Kubernetes
does not automatically configure Spark on Kubernetes.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4618
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <m...@apache.org>
Closes #3648 from Leemoonsoo/fix_spark_on_kubernetes and squashes the
following commits:
5dfe0010d [Lee moon soo] fix spark on kubernetes
---
k8s/zeppelin-server.yaml | 6 ++++--
spark/interpreter/src/main/resources/interpreter-setting.json | 2 +-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index ecb7606..348e1cd 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -28,7 +28,7 @@ data:
# Default value is 'local.zeppelin-project.org' while it points 127.0.0.1
and `kubectl port-forward zeppelin-server` will give localhost to connects.
# If you have your ingress controller configured to connect to
`zeppelin-server` service and have a domain name for it (with wildcard
subdomain point the same address), you can replace serviceDomain field with
your own domain.
serviceDomain: local.zeppelin-project.org:8080
- sparkContainerImage: spark:2.4.0
+ sparkContainerImage: spark:2.4.5
nginx.conf: |
daemon off;
worker_processes auto;
@@ -118,8 +118,10 @@ spec:
configMapKeyRef:
name: zeppelin-server-conf
key: serviceDomain
- - name: MASTER # default value of master property for spark interpreter.
+ - name: MASTER # default value of 'master' property for spark
interpreter.
value: k8s://https://kubernetes.default.svc
+ - name: SPARK_HOME # default value of 'SPARK_HOME' property for spark
interpreter.
+ value: /spark
# volumeMounts:
# - name: zeppelin-server-notebook-volume # configure this to persist
notebook
# mountPath: /zeppelin/notebook
diff --git a/spark/interpreter/src/main/resources/interpreter-setting.json
b/spark/interpreter/src/main/resources/interpreter-setting.json
index 100c97c..d8549b3 100644
--- a/spark/interpreter/src/main/resources/interpreter-setting.json
+++ b/spark/interpreter/src/main/resources/interpreter-setting.json
@@ -13,7 +13,7 @@
"type": "string"
},
"master": {
- "envName": "",
+ "envName": "MASTER",
"propertyName": "spark.master",
"defaultValue": "local[*]",
"description": "Spark master uri. ex) spark://master_host:7077",
