This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 3b234cfab6c [performance](exec) Performance problem create too many
scanner task (#38460)
3b234cfab6c is described below
commit 3b234cfab6ccbc3a1e739d139020bcfb1ef4503b
Author: HappenLee <happen...@hotmail.com>
AuthorDate: Wed Jul 31 14:34:01 2024 +0800
[performance](exec) Performance problem create too many scanner task
(#38460)
## Proposed changes
cherry pick the pr: #38430
<!--Describe your changes.-->
---
be/src/pipeline/exec/scan_operator.cpp | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/be/src/pipeline/exec/scan_operator.cpp
b/be/src/pipeline/exec/scan_operator.cpp
index 7322b085564..61543a170fc 100644
--- a/be/src/pipeline/exec/scan_operator.cpp
+++ b/be/src/pipeline/exec/scan_operator.cpp
@@ -1268,9 +1268,8 @@ Status ScanLocalState<Derived>::_start_scanners(
state(), this, p._output_tuple_desc, p.output_row_descriptor(),
scanners, p.limit(),
state()->scan_queue_mem_limit(), _scan_dependency,
// 1. If data distribution is ignored , we use 1 instance to scan.
- // 2. Else if this operator is not file scan operator, we use
config::doris_scanner_thread_pool_thread_num scanners to scan.
- // 3. Else, file scanner will consume much memory so we use
config::doris_scanner_thread_pool_thread_num / query_parallel_instance_num
scanners to scan.
- p.ignore_data_distribution() || !p.is_file_scan_operator()
+ // 2. Else, file scanner will consume much memory so we use
config::doris_scanner_thread_pool_thread_num / query_parallel_instance_num
scanners to scan.
+ p.ignore_data_distribution() && !p.is_file_scan_operator()
? 1
: state()->query_parallel_instance_num());
return Status::OK();
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
