eldenmoon opened a new pull request, #68969:
URL: https://github.com/apache/doris/pull/68969
### What problem does this PR solve?
SEARCH is declared non-nullable, but its internal Boolean execution
propagates field NULLs as SQL UNKNOWN. This makes NOT omit NULL documents,
skips the full AST when every referenced path is missing, and can produce
different behavior with score() or cached results.
Use two-valued document matching inside SEARCH in both parser modes:
- NULL and proven missing terms do not match; internal NOT complements the
matching document set.
- Preserve field:* as a field-existence leaf while leaving unscoped * as
match-all.
- Keep unavailable indexes explicit, preserve missing-field ASTs, and
isolate the new single-bitmap DSL cache with independent ownership.
- Allow Lucene pure NOT scoring to collect statistics around its fieldless
match-all node.
Ordinary SQL comparisons and MATCH keep their existing three-valued logic.
Parser precedence and Lucene modifier rules are preserved.
### Release note
SEARCH uses two-valued matching in both modes. NULL and proven missing
fields do not match ordinary terms; internal NOT includes non-matching
documents. field:* excludes NULL/missing fields. FE and BE must be upgraded
together for the complete contract.
### Check List (For Author)
- Test:
- ASAN FE/BE builds through build.sh passed with production
GLIBC_COMPATIBILITY=ON.
- Both supplied SEARCH contract suites passed, covering NULL/missing, both
modes, cache cold/warm, score ASC/DESC, and outer SQL/MATCH boundaries.
- Existing SEARCH/MATCH NULL regression suites: 2/2 passed.
- SearchDslParserTest: 169 passed. Focused SEARCH and
collection-statistics ASAN BE tests: 238 passed.
- BE UT used GLIBC_COMPATIBILITY=OFF only to work around duplicate
__gai_sigqueue definitions from repository/toolchain compatibility libraries;
no product configuration change.
- clang-format 16, build hygiene, and clang-tidy checks on changed lines
passed.
- Behavior changed: Yes, the explicitly selected SEARCH Boolean/NULL
contract above.
- Does this need documentation: Yes, SEARCH NULL/NOT/field-existence
semantics; the release note states the migration.
The full five-suite attachment was executed: the two SEARCH contract suites
passed; the three separate SQL CSE, score CAST/TRY_CAST, and AGG residual/TopN
suites still reproduce their existing failures. Those implementations and test
expectations are outside this PR's requested scope, and those three reproducers
are not added to CI by this PR. Unknown-type VARIANT, numeric BKD, and
NestedGroup BM25 scoring support is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
