924060929 opened a new pull request, #68468:
URL: https://github.com/apache/doris/pull/68468
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
`RewriteSearchToSlots` is the only rule that converts the raw `Search`
scalar function into a
`SearchExpression` with bound slot children, which is what lets BE evaluate
the predicate through
the inverted index. The whole-tree instance of that rule is registered inside
`notTraverseChildrenOf(ImmutableSet.of(LogicalCTEAnchor.class))`, so it does
not descend into
`LogicalCTEAnchor` subtrees. When a statement contains a CTE referenced more
than
`inline_cte_referenced_threshold` times (default 1), `CTEInline` keeps the
anchor materialized and
`PullUpCteAnchor` hoists all anchors above the whole plan. A SEARCH
predicate then sits below an
anchor, is never rewritten, reaches BE as `FunctionSearch`, and is rejected
with
`only inverted index queries are supported`.
Minimal reproduction (table `t` has a VARCHAR column `title` with an `USING
INVERTED` index):
WITH c AS (SELECT id FROM t WHERE search('title:hello'))
SELECT a.id FROM c a JOIN c b ON a.id = b.id;
`c` is referenced twice, so it is materialized instead of inlined. After
rewrite the producer still
contains the raw `search('title:hello')` filter and the scan has no
operative column, so the query
fails on BE. The same happens when the SEARCH is written in a
once-referenced (inlined) CTE while
any sibling CTE is materialized.
Fix:
Also register `RewriteSearchToSlots` in
`CTE_CHILDREN_REWRITE_JOBS_BEFORE_SUB_PATH_PUSH_DOWN`,
before `QueryColumnCollector` and before
`ColumnPruning`/`OperativeColumnDerive` of the
after-push-down job list. SEARCH predicates in CTE subtrees and in a main
body below a hoisted
anchor are then rewritten to `SearchExpression` the same way as the regular
whole-tree path.
### Release note
Fixed: a query that combines the `SEARCH` function with a materialized CTE
(a CTE referenced more
than `inline_cte_referenced_threshold` times) no longer fails with
`only inverted index queries are supported`.
### Check List (For Author)
- Test: Unit Test
- `SearchCteRewriteTest` (new): SEARCH inside a materialized CTE, and
SEARCH inside an inlined
CTE with a materialized sibling. Both fail without the fix and pass
with it.
- `RewriteSearchToSlotsTest` and `CheckSearchUsageTest` still pass.
- Behavior changed: Yes. SEARCH under a materialized CTE now plans to the
inverted-index path
instead of failing at execution. A SEARCH on a column without an inverted
index inside a CTE now
fails at planning time with the same clear error as the non-CTE path.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
