Gabriel39 opened a new pull request, #70319: URL: https://github.com/apache/doris/pull/70319
### What problem does this PR solve? Backport three master fixes to `branch-4.1`: avoid unnecessary database-object loading for SCHEMATA scans, prevent ORC condition-cache seek crashes or missing rows, and preserve JSON/WAL row counts when no file columns are projected. The commits retain their original authors and are applied in master merge order: 1. https://github.com/apache/doris/pull/70010 — Avoid database-object enumeration when SCHEMATA only needs names. 2. https://github.com/apache/doris/pull/70079 — Advance safely past SARG-filtered ORC stripe suffixes during condition-cache seeks. 3. https://github.com/apache/doris/pull/70154 — Track logical JSON records independently of projected columns and capture WAL row counts before column ownership is transferred. Branch adaptations: - Keep Thrift field 7 reserved and add the optional `include_db_ids` flag at upstream field 8 with its default `true`. Preserve existing privilege checks, pattern matching and schema-name formatting for both enumeration paths. - Use the upstream fixed ORC dependency revision `5962bb071d6345cc29c5353d64d85a203525e4d9`, which descends from the branch's existing revision. - Resolve the JSON reader conflicts by retaining 4.1's error handling and adding logical row accounting without importing unrelated missing-field/load-rejection changes. Remove the rooted-array pre-count that consumes the ondemand cursor before JSON-path materialization; retain the upstream rooted-array coverage. ### Validation - The branch's FE production and test sources compiled with the existing Java 8 target settings. All **7 targeted FE tests passed**, including the existing `FrontendServiceImplTest#testGetDBNames`, the 100,000-namespace case, filtering and legacy-request compatibility. FE Checkstyle passed. - A fresh ASAN build of the exact pinned ORC dependency passed **56 focused tests**, including **32 filtered-stripe-suffix boundary cases**. - All **7 affected C++ implementation/test/generator translation units compiled**. Clang-format 16 passed for all **8 affected C++ source/header files**. - The ORC fixture generator reproduced the committed fixture byte for byte. All **3 added Groovy suites compiled**. - Full Doris BE unit-test execution and external SQL regression execution remain for CI; they were not run locally for this backport. ### Release note Improve SCHEMATA enumeration for large catalogs. Fix potential crashes or missing rows during ORC condition-cache seeks and incorrect JSON/WAL row counts with empty file projections. ### Check List (For Author) - Test - [x] Unit tests and focused dependency tests - [x] Regression tests included and compiled; full SQL execution pending CI - Behavior changed: - [x] Yes. Apply the upstream fixes while preserving branch-specific validation and protocol compatibility. - Does this need documentation? - [x] No. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
