Gabriel39 opened a new pull request, #65499:
URL: https://github.com/apache/doris/pull/65499
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: FileScannerV2 treated a missing optional Parquet
`null_count` as proof that a row group contained no NULL values. When min/max
statistics were present, `IS NULL` predicates could therefore prune a row group
that actually contained NULL rows. This change treats unknown null counts
conservatively as possibly containing NULL while retaining available min/max
statistics, and covers the metadata combination with a unit test.
### Release note
Fix incorrect `IS NULL` results for Parquet files whose writers omit
`null_count` statistics.
### Check List (For Author)
- Test: Unit Test
- `ParquetStatistics*` BE unit tests (7 tests, 0 failures)
- `build-support/check-format.sh`
- `build-support/run-clang-tidy.sh --build-dir be/ut_build_ASAN`
- Behavior changed: Yes, row groups with unknown Parquet null counts are no
longer discarded by `IS NULL` pruning
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
