github-actions[bot] commented on code in PR #65449:
URL: https://github.com/apache/doris/pull/65449#discussion_r3558654180
##########
be/src/format_v2/table_reader.cpp:
##########
@@ -764,6 +777,70 @@ Status TableReader::prepare_split(const SplitReadOptions&
options) {
return _parse_delete_predicates(options);
}
+Status TableReader::_evaluate_partition_prune_conjuncts(const
VExprContextSPtrs& conjuncts,
+ bool* can_filter_all) {
+ DORIS_CHECK(can_filter_all != nullptr);
+ SCOPED_TIMER(_profile.runtime_filter_partition_prune_timer);
+ *can_filter_all = false;
+ if (conjuncts.empty() || _partition_values.empty()) {
+ return Status::OK();
+ }
+
+ VExprContextSPtrs partition_conjuncts;
+ for (const auto& conjunct : conjuncts) {
+ DORIS_CHECK(conjunct != nullptr);
+ DORIS_CHECK(conjunct->root() != nullptr);
+ std::set<GlobalIndex> global_indices;
+ collect_global_indices(conjunct->root(), &global_indices);
+ if (global_indices.empty()) {
+ continue;
+ }
+ const bool partition_only = std::ranges::all_of(global_indices,
[&](GlobalIndex index) {
+ if (index.value() >= _projected_columns.size()) {
Review Comment:
This partition-only test is too broad for split-level execution.
`collect_global_indices()` ignores zero-slot subexpressions, so a predicate
such as `p = 1 OR random() < 0.5` has only the partition slot in
`global_indices` and passes this check. For a split where `p != 1`, the new
code evaluates `random()` once in `prepare_split()` and may prune the entire
file, while the normal scan predicate would evaluate the volatile function for
each row. Doris already treats these functions as non-deterministic
(`VectorizedFnCall::is_deterministic()`), and similar planner rules avoid
moving volatile expressions for exactly this reason. Please exclude
non-deterministic / not-safe-to-pre-execute predicates here, or restrict this
split-level path to runtime-filter predicates that are safe to evaluate once
per partition value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
