airborne12 opened a new pull request, #69051:
URL: https://github.com/apache/doris/pull/69051
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: A SEARCH query can match rows through a non-tokenized
inverted index, while the same query with score() and descending top-k ordering
can return no rows. The selected index has no norms and contributes zero
tokens, leaving BM25 with a zero average document length and invalid scores
that can be dropped during top-k collection. Validate the physical index
selected for SEARCH scoring and return an explicit unsupported error when it is
not analyzed. Keep analyzed-index scoring and ordinary SEARCH filtering intact.
### Release note
SEARCH queries using score() now report an unsupported-index error when the
selected inverted index is not analyzed.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test: `GLIBC_COMPATIBILITY=OFF ./run-be-ut.sh --run
--filter=CollectionStatisticsTest.* -j 24` (65 passed). The local toolchain has
duplicate compatibility symbols when that option is on.
- [ ] Manual test
- [ ] No need to test or manual test
A SEARCH SQL regression suite was added for descending top-k, disabled WAND,
and ascending order. It was not run locally because no isolated Doris cluster
was started. clang-format, check-format, and build hygiene passed. clang-tidy
could not complete because of an existing unmatched NOLINTEND in
`be/src/core/types.h`.
- Behavior changed:
- [ ] No.
- [x] Yes. Scoring on a non-tokenized SEARCH index now returns a clear
error.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
