This is an automated email from the ASF dual-hosted git repository.
lingmiao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 796f44b [Bug] Fix bug that routine load blocked with TOO_MANY_TASKS
error (#4861)
796f44b is described below
commit 796f44beacd30ec8d300b483ab55193a2afeba7a
Author: Mingyu Chen <morningman....@gmail.com>
AuthorDate: Thu Nov 12 10:05:10 2020 +0800
[Bug] Fix bug that routine load blocked with TOO_MANY_TASKS error (#4861)
When receiving empty msg from kafka, the load process will quit abnormally.
Fix #4860
---
be/src/runtime/routine_load/data_consumer.cpp | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/be/src/runtime/routine_load/data_consumer.cpp
b/be/src/runtime/routine_load/data_consumer.cpp
index 32240f1..4b3954a 100644
--- a/be/src/runtime/routine_load/data_consumer.cpp
+++ b/be/src/runtime/routine_load/data_consumer.cpp
@@ -193,7 +193,11 @@ Status KafkaDataConsumer::group_consume(
consumer_watch.stop();
switch (msg->err()) {
case RdKafka::ERR_NO_ERROR:
- if (!queue->blocking_put(msg.get())) {
+ if (msg->len() == 0) {
+ // ignore msg with length 0.
+ // put empty msg into queue will cause the load process
shutting down.
+ break;
+ } else if (!queue->blocking_put(msg.get())) {
// queue is shutdown
done = true;
} else {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
