[doris] branch master updated: [improvement](fetch data) Merge result into batch to reduce rpc times (#17828)

[lihaopeng]

This is an automated email from the ASF dual-hosted git repository.

lihaopeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
     new 0da2cf270a [improvement](fetch data) Merge result into batch to reduce 
rpc times (#17828)
0da2cf270a is described below

commit 0da2cf270a686875436e3e11f47e81282b45f0f6
Author: ZenoYang <cookie...@qq.com>
AuthorDate: Sun Apr 23 15:07:28 2023 +0800

    [improvement](fetch data) Merge result into batch to reduce rpc times 
(#17828)
---
 be/src/runtime/buffer_control_block.cpp | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/be/src/runtime/buffer_control_block.cpp 
b/be/src/runtime/buffer_control_block.cpp
index 7565e3edac..c846844440 100644
--- a/be/src/runtime/buffer_control_block.cpp
+++ b/be/src/runtime/buffer_control_block.cpp
@@ -128,8 +128,18 @@ Status 
BufferControlBlock::add_batch(std::unique_ptr<TFetchDataResult>& result)
     }
 
     if (_waiting_rpc.empty()) {
+        // Merge result into batch to reduce rpc times
+        if (!_batch_queue.empty() &&
+            ((_batch_queue.back()->result_batch.rows.size() + num_rows) < 
_buffer_limit) &&
+            !result->eos) {
+            std::vector<std::string>& back_rows = 
_batch_queue.back()->result_batch.rows;
+            std::vector<std::string>& result_rows = result->result_batch.rows;
+            back_rows.insert(back_rows.end(), 
std::make_move_iterator(result_rows.begin()),
+                             std::make_move_iterator(result_rows.end()));
+        } else {
+            _batch_queue.push_back(std::move(result));
+        }
         _buffer_rows += num_rows;
-        _batch_queue.push_back(std::move(result));
         _data_arrival.notify_one();
     } else {
         auto ctx = _waiting_rpc.front();


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org