airborne12 opened a new pull request, #69600: URL: https://github.com/apache/doris/pull/69600
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: MATCH_REGEXP, wildcard, suffix and contains patterns without a literal prefix make SniiIndexSource::expand_terms enumerate the whole dictionary of every segment. LogicalIndexReader::visit_prefix_terms always passed an empty accept_key to DictBlockReader::visit_prefix_range, so the body of every dictionary entry was decoded (decode_dict_entry_rest) and the caller then rejected almost all of them with the required-text test and the pattern match. DictBlockReader already supports a key-only accept_key that skips the body of a rejected term. visit_prefix_terms now takes an optional accept_key, and expand_terms passes its term test (required text, then pattern match) as that predicate. Terms are tested exactly as before, only earlier, and the visitor still sees only accepted terms, so max_expansions and early stop count the same terms. Other callers keep the default (accept all). Measured on 99M AgentLogs rows (722 segments, tenant and status filters, token regexp 'transi.*', hot median): SNII 6028 ms -> 945 ms, V3 unchanged at about 590 ms. Result counts are identical to V3 for regexp, leading-wildcard regexp and phrase-prefix queries. Patterns with a literal prefix (for example '^transi.*') seek into the dictionary and are unchanged: SNII and V3 are level at about 100 ms before and after. Gdb sampling showed 58% of the time in decode_dict_entry_rest before and about 5% after. ### Release note Faster MATCH_REGEXP / wildcard term expansion on SNII inverted indexes. ### Check List (For Author) - Test: Unit Test (SniiRegexpQueryTest.AcceptKeySkipsBodiesOfRejectedTerms, SniiRegexpQueryTest.UnanchoredRegexpDecodesOnlyMatchedBodies; SniiB2T07Dict* and SniiRegexpQueryTest* pass on branch-4.2). Manual test (count equality against V3 on 99M rows, before/after timing). - Behavior changed: No - Does this need documentation: No 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
