hubgeter opened a new pull request, #70099:
URL: https://github.com/apache/doris/pull/70099
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
JSON external-table queries can return zero rows when column pruning removes
every physical file column, for example `COUNT(*)` with a partition predicate.
The V2 JSON reader derives its row count from a zero-column `Block`, so
accepted JSON records are lost and `TableReader` cannot materialize their
partition values.
Track accepted records independently of materialized columns, including
batch progress, malformed-record replacement and rollback. Preserve JSONPath
record selection when no file columns are requested. Add coverage for empty
projections and partition-only materialization across file splits.
### Release note
Fix incorrect zero-row results for JSON external-table queries with no
projected file columns.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- 32 focused ASAN JSON reader, split and prefetch tests passed
through `run-be-ut.sh`, including seven new test cases. The local build
selected the required test sources; the full BE unit suite was not run.
- Coverage includes empty/pruned schemas, arrays, JSONPaths,
malformed records, rollback, batch boundaries, EOF, row statistics and
partition-only scans across two splits with NONE/COUNT fallback.
- clang-format 16, build hygiene and changed-line clang-tidy passed.
- External Hive end-to-end validation was not run.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. JSON scans preserve accepted record counts when all file
columns are pruned.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
