Gabriel39 opened a new pull request, #65446: URL: https://github.com/apache/doris/pull/65446
### What problem does this PR solve? Issue Number: None Jira: [DORIS-27015](http://39.106.86.136:8090/browse/DORIS-27015) Problem Summary: Format v2 had two timestamp decoding inconsistencies: 1. Parquet INT96 represents a wall-clock timestamp without timezone semantics, but the reader passed the session timezone into decoding and shifted values (for example, by +8 hours in Asia/Shanghai). 2. ORC timestamps with nanosecond precision were truncated to microseconds. This disagreed with the expected half-up rounding behavior and could also make row decoding, stripe statistics, and SARG predicate pushdown use different boundaries. This PR: - decodes Parquet INT96 as a timezone-independent wall-clock value; - rounds ORC nanoseconds to microseconds with second carry for both DATETIMEV2 and TIMESTAMPTZ; - applies the same rounding to ORC statistics; - expands timestamp SARG comparisons to the exact nanosecond intervals that round to each Doris microsecond, with conservative handling for pre-epoch statistics; - adds format v2 unit coverage for timezone behavior, rounding, carry, negative timestamps, and timestamp SARG pruning. ### Release note Fix timestamp timezone handling and nanosecond rounding in format v2 Parquet and ORC readers. ### Check List (For Author) - Test: Unit Test - On `[email protected]:/mnt/disk3/gabriel/Workspace/dev2/doris`: - 11 targeted format v2 Parquet/ORC BE unit tests passed. - clang-format 16 and `build-support/check-format.sh` passed. - clang-tidy was attempted with the remote ASAN UT compilation database; the remote LLVM/GCC sysroot is missing `stddef.h`. Extracted diagnostics contained no warnings on this PR's changed lines. - Behavior changed: Yes - Parquet INT96 no longer shifts with the session timezone. - ORC nanoseconds are rounded to microseconds instead of truncated. - Does this need documentation: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
