This is an automated email from the ASF dual-hosted git repository.
jianliangqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 879a729afb [improve](inverted index) not apply inverted index on 'in'
or 'not_in' predicate which is produced by runtime_filter (#16952)
879a729afb is described below
commit 879a729afb481e92352f8549c3c7cdf95e045922
Author: YueW <45946325+tany...@users.noreply.github.com>
AuthorDate: Tue Feb 21 14:24:50 2023 +0800
[improve](inverted index) not apply inverted index on 'in' or 'not_in'
predicate which is produced by runtime_filter (#16952)
When there are multi-table join query, there will be many in or not_in
predicate of runtime filter pushed down to the storage layer. According to our
test, if apply those predicates by inverted index, the performance will be
degraded because there are many conditions in in_predicate. Therefore, the
inverted index not apply on in or not_in predicate which is produced by
runtime_filter.
Based on that situation, this pr will do:
not apply inverted index on in or not_in predicate which is produced by
runtime_filter.
---
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
index 39ada1e7d5..e6f40250b1 100644
--- a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
+++ b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
@@ -686,11 +686,14 @@ Status
SegmentIterator::_apply_inverted_index_on_column_predicate(
_inverted_index_iterators[unique_id] == nullptr ||
(pred->type() != PredicateType::MATCH && handle_by_fulltext) ||
pred->type() == PredicateType::IS_NULL || pred->type() ==
PredicateType::IS_NOT_NULL ||
- pred->type() == PredicateType::BF) {
+ pred->type() == PredicateType::BF ||
+ ((pred->type() == PredicateType::IN_LIST || pred->type() ==
PredicateType::NOT_IN_LIST) &&
+ pred->predicate_params()->marked_by_runtime_filter)) {
// 1. this column no inverted index
// 2. equal or range for fulltext index
// 3. is_null or is_not_null predicate
// 4. bloom filter predicate
+ // 5. in_list or not_in_list predicate produced by runtime filter
remaining_predicates.emplace_back(pred);
} else {
roaring::Roaring bitmap = _row_bitmap;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
