HappenLee commented on code in PR #9792:
URL: https://github.com/apache/incubator-doris/pull/9792#discussion_r884111183
##########
be/src/exec/olap_scanner.cpp:
##########
@@ -327,6 +327,7 @@ Status OlapScanner::get_batch(RuntimeState* state,
RowBatch* batch, bool* eof) {
break;
}
+ SCOPED_TIMER(_parent->_eval_conjuctx_timer);
Review Comment:
should not do timer each rows, could cause performance problem
##########
be/src/olap/column_predicate.h:
##########
@@ -91,6 +91,9 @@ class ColumnPredicate {
protected:
uint32_t _column_id;
bool _opposite;
+ mutable uint64_t _evaluated_rows = 1;
Review Comment:
the value only use in bloom filter predicate. it should not define here?
##########
be/src/vec/exec/volap_scan_node.cpp:
##########
@@ -554,8 +576,11 @@ Block* VOlapScanNode::_alloc_block(bool& get_free_block) {
int VOlapScanNode::_start_scanner_thread_task(RuntimeState* state, int
block_per_scanner) {
std::list<VOlapScanner*> olap_scanners;
int assigned_thread_num = _running_thread;
- size_t max_thread = std::min(_volap_scanners.size(),
-
static_cast<size_t>(config::doris_scanner_thread_pool_thread_num));
+ size_t max_thread = config::doris_scanner_queue_size;
Review Comment:
If this code change is not sure to be effective, I think it can not be
modified first
##########
be/src/olap/bloom_filter_predicate.h:
##########
@@ -158,6 +160,16 @@ void
BloomFilterColumnPredicate<T>::evaluate(vectorized::IColumn& column, uint16
new_size += _specific_filter->find_olap_engine(cell_value);
}
}
+ // If the pass rate is very high, for example > 50%, then the bloomfilter
is useless.
+ // Some bloomfilter is useless, for example ssb 4.3, it consumes a lot of
cpu but it is
+ // useless.
+ _evaluated_rows += *size;
+ _passed_rows += new_size;
+ if (_evaluated_rows > 100) {
Review Comment:
`100` should be a config of BE
##########
be/src/exec/olap_scan_node.cpp:
##########
@@ -689,11 +692,11 @@ Status OlapScanNode::build_scan_key() {
return Status::OK();
}
-static Status get_hints(TabletSharedPtr table, const TPaloScanRange&
scan_range,
- int block_row_count, bool is_begin_include, bool
is_end_include,
- const std::vector<std::unique_ptr<OlapScanRange>>&
scan_key_range,
- std::vector<std::unique_ptr<OlapScanRange>>*
sub_scan_range,
- RuntimeProfile* profile) {
+Status OlapScanNode::get_hints(TabletSharedPtr table, const TPaloScanRange&
scan_range,
Review Comment:
why change the code to not static
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
