liaoxin01 commented on code in PR #48171:
URL: https://github.com/apache/doris/pull/48171#discussion_r1981154326
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadStatistic.java:
##########
@@ -61,6 +61,10 @@ public class RoutineLoadStatistic {
public long committedTaskNum = 0;
@SerializedName(value = "abortedTaskNum")
public long abortedTaskNum = 0;
+ @SerializedName(value = "currentCommittedTaskNum")
Review Comment:
no need to serialize.
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java:
##########
@@ -136,6 +140,39 @@ public int getTotalMaxConcurrentTaskNum() {
return beIdToMaxConcurrentTasks.values().stream().mapToInt(i ->
i).sum();
}
+ public Map<Long, String> getAbnormalJobs() {
+ return abnormalJobs;
+ }
+
+ public void cleanAbnormalJobs() {
+ abnormalJobs.clear();
+ }
+
+ public void addAbnormalJob(Long id, String reason) {
+ if (!abnormalJobs.containsKey(id)) {
+ MetricRepo.COUNTER_ROUTINE_LOAD_ABNORMAL_JOB_NUMS.increase(1L);
+ }
+ abnormalJobs.put(id, reason);
+ }
+
+ public void removeAbnormalJob(Long id) {
+ if (!abnormalJobs.containsKey(id)) {
+ return;
+ }
+ abnormalJobs.remove(id);
+ long value =
MetricRepo.COUNTER_ROUTINE_LOAD_ABNORMAL_JOB_NUMS.getValue() - 1;
+ MetricRepo.COUNTER_ROUTINE_LOAD_ABNORMAL_JOB_NUMS.update(value);
Review Comment:
COUNTER_ROUTINE_LOAD_ABNORMAL_JOB_NUMS can be obtained by size of
abnormalJobs.
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -1277,6 +1277,30 @@ public class Config extends ConfigBase {
@ConfField(mutable = true, masterOnly = true)
public static int max_get_kafka_meta_timeout_second = 60;
+ /**
+ * The interval (in seconds) to check for abnormal routine load jobs,
multiplied by maxBatchInterval.
+ */
+ @ConfField(mutable = true, masterOnly = true)
+ public static int abnormal_check_interval_multiplier = 50;
Review Comment:
50 maybe too large.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
