This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 43f80e2633 [enhancement](load) Increase batch size of node channel to
improve import performance (#13912)
43f80e2633 is described below
commit 43f80e263377d07429fb082ed9ade746b4d910bd
Author: Xin Liao <liaoxin...@126.com>
AuthorDate: Fri Nov 11 18:05:36 2022 +0800
[enhancement](load) Increase batch size of node channel to improve import
performance (#13912)
---
be/src/vec/sink/vtablet_sink.cpp | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/be/src/vec/sink/vtablet_sink.cpp b/be/src/vec/sink/vtablet_sink.cpp
index a6e4a96566..3599abcbb3 100644
--- a/be/src/vec/sink/vtablet_sink.cpp
+++ b/be/src/vec/sink/vtablet_sink.cpp
@@ -68,6 +68,12 @@ Status VNodeChannel::init(RuntimeState* state) {
_cur_add_block_request.set_eos(false);
_name = fmt::format("VNodeChannel[{}-{}]", _index_channel->_index_id,
_node_id);
+ // The node channel will send _batch_size rows of data each rpc. When the
+ // number of tablets is large, the number of data rows received by each
+ // tablet is small, TabletsChannel need to traverse each tablet for import.
+ // so the import performance is poor. Therefore, we set _batch_size to
+ // a relatively large value to improve the import performance.
+ _batch_size = std::max(_batch_size, 8192);
return Status::OK();
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
