yujun777 commented on code in PR #22889:
URL: https://github.com/apache/doris/pull/22889#discussion_r1434811587
##########
fe/fe-core/src/main/java/org/apache/doris/load/sync/SyncJobManager.java:
##########
@@ -283,13 +286,28 @@ public void updateNeedSchedule() throws UserException {
// Stopped jobs will be removed after Config.label_keep_max_second.
public void cleanOldSyncJobs() {
LOG.debug("begin to clean old sync jobs ");
+ cleanFinishedSyncJobsIf(job ->
job.isExpired(System.currentTimeMillis()), -1);
+ }
+
+ /**
+ * Remove completed jobs if total job num exceed Config.label_num_threshold
+ */
+ public void cleanOverLimitSyncJobs() {
+ if (idToSyncJob.size() <= Config.label_num_threshold) {
+ return;
+ }
+ LOG.debug("begin to clean finished sync jobs ");
+ cleanFinishedSyncJobsIf(SyncJob::isCompleted,
Config.label_num_threshold);
Review Comment:
Is it better to remove oldest jobs when exceed limit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
