CalvinKirs commented on code in PR #24035:
URL: https://github.com/apache/doris/pull/24035#discussion_r1319327028
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java:
##########
@@ -811,24 +820,30 @@ private void updateNumOfData(long numOfTotalRows, long
numOfErrorRows, long unse
.add("current_total_rows",
this.jobStatistic.currentTotalRows)
.add("current_error_rows",
this.jobStatistic.currentErrorRows)
.add("max_error_num", maxErrorNum)
+ .add("max_filter_ratio", maxFilterRatio)
.add("msg", "reset current total rows and current
error rows "
- + "when current total rows is more than base")
+ + "when current total rows is more than base
or the filter ratio is more than the max")
.build());
}
// reset currentTotalNum and currentErrorNum
this.jobStatistic.currentErrorRows = 0;
this.jobStatistic.currentTotalRows = 0;
- } else if (this.jobStatistic.currentErrorRows > maxErrorNum) {
+ } else if (this.jobStatistic.currentErrorRows > maxErrorNum
+ || (this.jobStatistic.currentTotalRows > 0
+ && (this.jobStatistic.currentErrorRows /
this.jobStatistic.currentTotalRows) > maxFilterRatio)) {
Review Comment:
This will lose a lot of precision, and even many times the result will be
wrong
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
