HappenLee opened a new pull request, #68097:
URL: https://github.com/apache/doris/pull/68097
### What problem does this PR solve?
Issue Number: None
Related PR: #68018
Problem Summary:
A single-version rowset stores a placeholder in `__DORIS_COMMIT_TSO_COL__`;
its logical value comes from the rowset's publish metadata. Some read paths
request a physical column reader before the data iterator supplies that value.
A cross-column condition such as `commit_tso > 0 OR v = 99` can therefore prune
a segment using the placeholder ZoneMap. Historical reads and JSONB-based
row-id/point reads can also return the placeholder.
Resolve the logical reader in `Segment::get_column_reader` using the source
rowset's version and commit TSO, before consulting physical metadata or the
cache. Single-version published reads receive a constant reader; multi-version
and unpublished internal reads retain physical readers. Keep only physical
readers in `ColumnReaderCache`, removing the optional constant parameter and
cache-upgrade state. Rename deliberate Variant/storage metadata accesses to
`get_physical_column_reader`.
Route segment/page expression ZoneMaps, ordinary predicates, predicate
simplification, aggregate statistics, data iterators and indexes through this
entry point. Supply source rowset metadata to historical and row-id reads. For
JSONB historical reads, two-phase fetch and short-circuit point queries,
preserve ordinary row-store columns and resolve TSO separately. Point queries
requesting TSO bypass the JSONB row cache because it does not retain the source
rowset location.
This is an alternative to #68018's cache-upgrade approach. It retains that
PR's same-key LRU replacement fix and constant-reader index no-op: concurrent
physical cache misses can still insert the same key, and physical indexes are
inapplicable to logical constants. No storage format or FE/BE protocol changes.
### Release note
Fix incorrect commit TSO values and filtering in expression pushdown,
historical reads, two-phase row-id fetch and point queries.
### Check List (For Author)
- Test
- [ ] Regression test: added `tso_p0/test_commit_tso_reader_paths`,
using runtime rowset metadata as the oracle; not executed because this worktree
has no independent FE/BE cluster.
- [x] Unit Test: 62 tests from 6 suites passed under ASAN_UT using
`run-be-ut.sh -j 48` (`SegmentIteratorExprZonemapTest`,
`ConstantColumnIteratorTest`, `ColumnReaderCacheTest`,
`HistoricalRowRetrieverTest`, `PointQueryCommitTsoTest`,
`RowIdStorageReaderTest`).
- [x] Manual test: clang-format 16 check, build hygiene and `git diff
--check` passed. BE sources compiled and `doris_be` linked; full `build.sh` was
blocked when the kuromoji dictionary generator hit an ASAN double-free in
OpenMP initialization. clang-tidy was attempted but blocked by the existing
unmatched NOLINT marker in `be/src/core/types.h:576`; it is not reported as
passed.
- Behavior changed:
- [x] Yes. Logical TSO values are used consistently before filtering and
when reading JSONB-backed rows; only point queries requesting TSO bypass the
row cache.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
