eldenmoon opened a new pull request, #69347:
URL: https://github.com/apache/doris/pull/69347
### What problem does this PR solve?
Related PR: #68969 (FE parser prerequisite), #68758
SEARCH declares a non-nullable result, but NULL/UNKNOWN leaves can produce
different match sets under NOT, scoring, or cached execution. This PR
implements the backend two-valued contract: ordinary terms do not match NULL or
proven-absent fields; AND/OR combine match sets; internal NOT complements them.
Ordinary SQL expressions and MATCH retain their existing three-valued semantics.
Filter physical NULL postings before Boolean composition, including NULL
ARRAY values with retained nested payloads. Preserve genuine term capabilities
for grouped opening, candidate propagation, and block-max WAND. Apply
exclusions after every conjunction fast path. Keep cached results independently
owned and preserve source-managed document IDs across collection paths.
A missing iterator is not proof that a Variant path is absent. If historical
rowsets contain the path but lack an executable index after CREATE INDEX,
SEARCH now fails explicitly instead of silently dropping matches. Tests write
actual objects with `parse_to_variant`, verify their type and extracted values,
then check the error, newly indexed rows, and genuinely absent paths. This
change does not backfill historical indexes or add a scan fallback.
**Merge order:** merge #68969 first. It preserves the field in `field:*` and
`field:(*)`; this PR supplies the backend non-NULL existence evaluation. Both
PRs target master. This PR is initially a draft because its field-scoped
wildcard SQL regressions require that FE prerequisite. It contains no FE source
or FE unit-test changes.
### Release note
SEARCH consistently uses two-valued matching for NULL and proven-absent
fields, including internal and outer SQL NOT. Field-scoped `*` tests non-NULL
existence. A required unavailable Variant index reports an error. Ordinary SQL
and MATCH NULL semantics remain unchanged.
### Check List (For Author)
- Test:
- [x] 353 targeted ASAN BE tests passed across 16 suites on the final
rebased backend tree; one existing disabled test. Backend source and tests are
byte-for-byte identical after separating the FE files.
- [x] The new conjunction fast-path case failed before the fix and passed
afterward. Nullable docs-only scoring, grouped term opening, candidates, WAND,
collectors, and the real historical Variant writer/segment case passed.
- [x] clang-format 16, build hygiene, and changed-line clang-tidy for the
conflict adaptation passed.
- [x] Six focused SQL suites passed on the earlier combined revision
`2318335d88d`: historical rowset indexes, missing doc paths, Variant
CLucene/SNII subcolumns, cross-field NULL, and the Variant two-valued contract.
The historical-rowset suite includes 96 expected-error checks and 37
runner-generated result sets.
- [ ] SQL regressions after this rebase and split; field-scoped wildcard
cases require #68969.
- Behavior changed: Yes, SEARCH consistently returns a two-valued match set
and rejects required unavailable indexes.
- Does this need documentation: Yes; the SEARCH regression semantics guide
is updated.
Local BE UT uses `GLIBC_COMPATIBILITY=OFF`. The earlier combined product
ASAN build used compatibility ON and passed the GLIBC_2.17 artifact check; it
was not rebuilt after this split. No latency benchmark, mixed-version run, or
enterprise NestedGroup execution was performed. CI results are pending.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases and FE prerequisite
- [ ] Confirm documentation
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
