englefly commented on PR #68314: URL: https://github.com/apache/doris/pull/68314#issuecomment-5790380744
Rebased onto the current `branch-4.2` (`c0cbcc3e49c`) — the PR is mergeable again. The branch is now a single commit `e044385682f` (was 4). **Why one commit instead of the previous three picks.** The prerequisites this PR carried (`#64535`, `#59263`) were only needed because branch-4.2 stopped just before them. `#59263` replaced the `#64535` `MetaPathStriper` design again within this same pick, and branch-4.2 has since gained its own meta-path series (`#68214`) plus the `#68331` predicate-path fix — replaying the dead intermediate state only reintroduces conflicts. The single commit contains the cumulative result of the three picks plus the review follow-ups. **Conflicts resolved** * `NestedColumnPruning.java` — branch-4.2's `#68331` reworked the predicate-path reconciliation while this branch was open. I kept branch-4.2's case-insensitive `coversPrefixIgnoreCase`/`isCoveredByAllPath` helpers, but kept master's `addPredicatePathsToFinalAllAccessPaths` policy of *retaining* metadata predicate paths instead of dropping them: the new BE derives `NULL_MAP_ONLY`/`OFFSET_ONLY` from the all access paths only, and only while no access path still requires payload (`ColumnIterator::_check_and_set_meta_read_mode`), so the extra NULL/OFFSET predicate-path cleanup `#68214`/`#68331` added to protect the older BE is not needed on top of the new BE. `PruneNestedColumnTest.testStructIsNullMixedAccess` pins this behaviour; if reviewers prefer to keep `#68331`'s stricter cleanup, that test's second case becomes `predicateAccessPaths = []`. * `PruneNestedColumnTest.java` — six branch-4.2 tests pinned the previous design and are replaced by ported counterparts that assert the new behaviour: `testFullFieldAccessStripsExactDataSkippingPath` → `testFullFieldAccessKeepsExactMetadataPath`, `testCardinalityMapElementOffsetCoveredByValueFieldAccess` → `testCardinalityMapElementOffsetPredicateStaysOutOfAllAccessPaths`, `testMapElementArrayNullPathCoveredByValueFieldAccess` → `testMapElementArrayNullPredicateStaysOutOfAllAccessPaths`, `testNonOlapDataSkippingOnlyAccessPathFallback` → `testNonOlapMetadataAccessPathFallback`; the two struct-root OFFSET cases are covered by `testMetadataPathBelowSameNamedStructField` and the map-star expansion tests. * `enable_prune_nested_column` stays on thrift field id `229` (226-228 are taken on branch-4.2). * Upstream edits made to `olap_scan_operator.cpp`, `olap_scanner.cpp`, `column_reader.cpp` and `SessionVariable.java` since the branch was created are preserved. **Verified on the rebased commit** * `./build.sh --fe` OK, checkstyle included. * FE: `run-fe-ut.sh --run 'PruneNestedColumnTest,VariantPruningLogicTest,TopNRuntimeFilterTest'` → 87 tests, 0 failures. * BE: `BUILD_TYPE_UT=Release run-be-ut.sh --run --filter='ColumnReaderTest.*:SegmentIteratorLazyPrunedTest.*:SlotDescriptorTest.*'` → 61 tests, 0 failures. The regression suites under `nereids_rules_p0/column_pruning` and `datatype_p0/complex_types` have not been re-run on the new base yet (`run buildall`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
