[GitHub] [incubator-doris] morningman commented on a change in pull request #4383: [SparkLoad]Use the yarn command to get status and kill the application

[GitBox]

morningman commented on a change in pull request #4383:
URL: https://github.com/apache/incubator-doris/pull/4383#discussion_r477199868



##########
File path: fe/fe-core/src/main/java/org/apache/doris/common/Config.java
##########
@@ -543,6 +543,14 @@
     @ConfField
     public static String spark_resource_path = "";
 
+    /**
+     * Default yarn client path
+     */
+    public static String yarn_client_path = PaloFe.DORIS_HOME_DIR + 
"/lib/yarn-client/hadoop/bin/yarn";

Review comment:
       @ConfField

##########
File path: 
fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkEtlJobHandler.java
##########
@@ -74,17 +67,21 @@
     private static final String JOB_CONFIG_DIR = "configs";
     private static final String ETL_JOB_NAME = "doris__%s";
     // 5min
-    private static final int GET_APPID_MAX_RETRY_TIMES = 300;
-    private static final int GET_APPID_SLEEP_MS = 1000;
-
-    class SparkAppListener implements Listener {
+    private static final long GET_APPID_TIMEOUT_MS = 300000L;
+    // 30s
+    private static final long EXEC_CMD_TIMEOUT_MS = 30000L;
+    // yarn command
+    private static final String YARN_STATUS_CMD = "%s --config %s application 
-status %s";
+    private static final String YARN_KILL_CMD = "%s --config %s application 
-kill %s";
+
+    class SparkAppListener implements SparkLoadAppHandle.Listener {

Review comment:
       Not used?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org