airborne12 opened a new pull request, #69637:
URL: https://github.com/apache/doris/pull/69637
### What problem does this PR solve?
Issue Number: None
Related PR: #69600
Problem Summary: Backport of #69600 to branch-4.2. MATCH_REGEXP, wildcard,
suffix and contains patterns without a literal prefix make
`SniiIndexSource::expand_terms` enumerate the whole dictionary of every
segment. `LogicalIndexReader::visit_prefix_terms` passed an empty `accept_key`,
so the body of every dictionary entry was decoded before the caller rejected
almost all of them. `visit_prefix_terms` now takes an optional `accept_key`,
and `expand_terms` passes its term test as that predicate, so a rejected term
is skipped before its body is decoded. Terms are tested exactly as before, only
earlier.
Measured on master's parent line (99M AgentLogs rows, 722 segments): token
regexp `transi.*` on SNII 6028 ms -> 945 ms, V3 about 590 ms.
### Branch adaptations
None. `git cherry-pick -x 6979f7fa1e` applied cleanly to the current
branch-4.2 and touches the same four files as the original:
`logical_index_reader.{h,cpp}`, `snii_index_source.cpp` and
`snii_query_test.cpp`.
### Validation
- The same change (before the one-line clang-format fix of the declaration
in `logical_index_reader.h`) was built and run on an earlier branch-4.2 base
(`ecbc5feab9`): `SniiB2T07Dict*` and `SniiRegexpQueryTest*`, 31 tests passed,
including the two new tests.
- On master, `Snii*` passed (1373 tests) and the master CI passed all checks.
- Not rebuilt locally on the current branch-4.2 tip; this PR's BE UT run
covers that.
### Release note
Faster MATCH_REGEXP / wildcard term expansion on SNII inverted indexes.
### Check List (For Author)
- Test: Unit Test
- Behavior changed: No
- Does this need documentation: No
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
