HappenLee opened a new pull request, #68446:
URL: https://github.com/apache/doris/pull/68446
### What problem does this PR solve?
Literal `IN` lookups across hash buckets use the normal scan path even when
a merge-on-write unique-key table stores complete rows. For example, looking up
a small set of addresses while returning wide JSON columns still initializes
scanners, locates key ranges, and reads individual columns.
This PR adds an opt-in batch point-query path that groups distinct keys by
tablet and reuses the existing point-query RPC and row-store reader. Eligible
queries have at most 100 string literals in a single `IN` predicate, direct
column/alias projections, and an unpartitioned MoW table with complete row
storage, light schema change, and one non-null `VARCHAR` primary key that is
also the sole hash distribution column. Unsupported query shapes retain normal
planning; existing equality and prepared-statement point queries retain their
current execution paths.
The FE sends asynchronous tablet requests with a configurable per-query
window, replica retries, one absolute deadline, cancellation, and an aggregate
result-size limit. The current scheduler consumes responses in submission
order. All requests use one visible partition version. The BE captures and pins
the corresponding rowsets and delete bitmap, bypasses the unversioned RowCache
for snapshot reads, and filters deleted rows correctly when a request contains
both deleted and live keys. The FE requires a matching snapshot acknowledgment
from every BE and returns rows only after all requests succeed; an older BE or
an oversized response produces an error instead of a partial or inconsistent
result.
### Release note
Add `enable_batch_point_query` (default `false`) for eligible literal `IN`
lookups on row-store tables. Add `batch_point_query_concurrency` (default
`100`, range `1`–`100`) to control the per-query tablet request window. Both FE
and BE must support the new snapshot lookup fields before enabling this
feature. Existing `enable_short_circuit_query` must also be enabled.
```sql
SET enable_batch_point_query = true;
SET batch_point_query_concurrency = 100;
SELECT address, out_edges_json, in_edges_json
FROM address_graph
WHERE address IN ('address_1', 'address_2', 'address_3');
```
### Check List (For Author)
- Test
- [x] Regression test: generated and replayed
`point_query_p0/test_batch_point_query`, covering the
nine-address/eight-column/128-bucket shape, 100 wide rows, duplicates, missing
keys, NULL values, mixed live/deleted rows, unsupported shapes, and session
concurrency settings.
- [x] Unit Test: 18 focused FE tests; 16 BE tests, including real MoW
snapshot fixtures, deletion filtering, cancellation, retries, routing,
configuration, and result limits.
- [x] Manual test: 100 atomic two-tablet updates with 2,771 concurrent
snapshot reads returned no mixed generations. Real SQL also checked session
isolation, rejected settings, and complete scan/batch result equivalence at
concurrency 1, 8, 16, and 100.
- FE build and Checkstyle passed. BE ASAN build, clang-format 16, and
build hygiene passed. Local ASAN service startup used `malloc_context_size=2`
for an existing PHDR unwinding startup issue; BE unit tests used standard
sanitizer options. Clang-tidy remains affected by pre-existing header/NOLINT
and complexity diagnostics.
- Validation was performed on the development checkout. The publication
branch removes an unrelated AI-adapter change from that checkout; all 13
feature/test files are byte-identical. The exact publication head has not been
rebuilt separately.
- A previous single-session RELEASE benchmark covered the
fixed-concurrency-8 version. A fresh benchmark for the configurable default-100
version is in progress; no performance result is claimed for default 100 yet.
- Behavior changed:
- [x] Yes. Eligible literal `IN` queries use snapshot row-store lookup
when explicitly enabled. The new switch defaults to disabled.
- Does this need documentation?
- [x] Yes. The opt-in scope, FE/BE version requirement, and concurrency
setting need documentation before release; usage is included above.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
