This is an automated email from the ASF dual-hosted git repository.
zjffdu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 5dedab9 [ZEPPELIN-4906]. FLINK_CONF_DIR doesn't work in yarn
interpreter mode
5dedab9 is described below
commit 5dedab9e507257fff428abf75a32b45536cb04ba
Author: Jeff Zhang <zjf...@apache.org>
AuthorDate: Wed Jun 24 10:48:10 2020 +0800
[ZEPPELIN-4906]. FLINK_CONF_DIR doesn't work in yarn interpreter mode
### What is this PR for?
This use env instead of properties for these flink related config. Because
the env will be reset in yarn interpreter mode
(https://github.com/apache/zeppelin/blob/master/zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java#L295)
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4906
### How should this be tested?
* Manually tested,
Here's what I see after this PR.
```
INFO [2020-06-24 10:46:06,461]
({FIFOScheduler-interpreter_583675831-Worker-1}
FlinkScalaInterpreter.scala[initFlinkConfig]:153) - FLINK_HOME:
/Users/jzhang/Temp/hadoop/yarn/usercache/jzhang/appcache/application_1591195707498_0116/container_1591195707498_0116_01_000001/flink
INFO [2020-06-24 10:46:06,462]
({FIFOScheduler-interpreter_583675831-Worker-1}
FlinkScalaInterpreter.scala[initFlinkConfig]:154) - FLINK_CONF_DIR:
/Users/jzhang/Temp/hadoop/yarn/usercache/jzhang/appcache/application_1591195707498_0116/container_1591195707498_0116_01_000001/flink/conf
INFO [2020-06-24 10:46:06,462]
({FIFOScheduler-interpreter_583675831-Worker-1}
FlinkScalaInterpreter.scala[initFlinkConfig]:155) - HADOOP_CONF_DIR:
/Users/jzhang/Java/lib/hadoop-2.7.7/etc/hadoop
INFO [2020-06-24 10:46:06,462]
({FIFOScheduler-interpreter_583675831-Worker-1}
FlinkScalaInterpreter.scala[initFlinkConfig]:156) - YARN_CONF_DIR:
/Users/jzhang/Java/lib/hadoop-2.7.7/etc/hadoop
INFO [2020-06-24 10:46:06,462]
({FIFOScheduler-interpreter_583675831-Worker-1}
FlinkScalaInterpreter.scala[initFlinkConfig]:157) - HIVE_CONF_DIR:
INFO [2020-06-24 10:46:06,464]
({FIFOScheduler-interpreter_583675831-Worker-1}
FlinkScalaInterpreter.scala[initFlinkConfig]:160) - Using flink: 1.10.0
INFO [2020-06-24 10:46:06,465]
({FIFOScheduler-interpreter_583675831-Worker-1} FlinkShims.java[loadShims]:51)
- Initializing shims for Flink 1.10
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjf...@apache.org>
Closes #3820 from zjffdu/ZEPPELIN-4906 and squashes the following commits:
e38cc301b [Jeff Zhang] [ZEPPELIN-4906]. FLINK_CONF_DIR doesn't work in yarn
interpreter mode
---
.../org/apache/zeppelin/flink/FlinkScalaInterpreter.scala | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git
a/flink/interpreter/src/main/scala/org/apache/zeppelin/flink/FlinkScalaInterpreter.scala
b/flink/interpreter/src/main/scala/org/apache/zeppelin/flink/FlinkScalaInterpreter.scala
index 027838e..1175b7b 100644
---
a/flink/interpreter/src/main/scala/org/apache/zeppelin/flink/FlinkScalaInterpreter.scala
+++
b/flink/interpreter/src/main/scala/org/apache/zeppelin/flink/FlinkScalaInterpreter.scala
@@ -145,11 +145,11 @@ class FlinkScalaInterpreter(val properties: Properties) {
}
private def initFlinkConfig(): Config = {
- val flinkHome = properties.getProperty("FLINK_HOME",
sys.env.getOrElse("FLINK_HOME", ""))
- val flinkConfDir = properties.getProperty("FLINK_CONF_DIR",
sys.env.getOrElse("FLINK_CONF_DIR", ""))
- val hadoopConfDir = properties.getProperty("HADOOP_CONF_DIR",
sys.env.getOrElse("HADOOP_CONF_DIR", ""))
- val yarnConfDir = properties.getProperty("YARN_CONF_DIR",
sys.env.getOrElse("YARN_CONF_DIR", ""))
- val hiveConfDir = properties.getProperty("HIVE_CONF_DIR",
sys.env.getOrElse("HIVE_CONF_DIR", ""))
+ val flinkHome = sys.env.getOrElse("FLINK_HOME", "")
+ val flinkConfDir = sys.env.getOrElse("FLINK_CONF_DIR", "")
+ val hadoopConfDir = sys.env.getOrElse("HADOOP_CONF_DIR", "")
+ val yarnConfDir = sys.env.getOrElse("YARN_CONF_DIR", "")
+ val hiveConfDir = sys.env.getOrElse("HIVE_CONF_DIR", "")
LOGGER.info("FLINK_HOME: " + flinkHome)
LOGGER.info("FLINK_CONF_DIR: " + flinkConfDir)
LOGGER.info("HADOOP_CONF_DIR: " + hadoopConfDir)
