Gabriel39 opened a new pull request, #68301:
URL: https://github.com/apache/doris/pull/68301
### What problem does this PR solve?
This ports #68297 to `master`, preserving the first of five planned
extractions from #67784.
Binary `Field` values can retain references to released source storage, and
Hive binary text needs its own Base64 contract. TIMESTAMPTZ output can lose
historical offset seconds, format invalid NULL payloads, or fail again while
reporting a boundary cast error.
- Own long binary Field values while keeping short values inline. Preserve
execution type lengths and decoder bytes, and add Hive Base64 and hexadecimal
decoding support.
- Explicitly reject unsupported binary hash keys, IN, aggregates, predicates
and computed partition transforms. Keep the existing FE comparison/group/join
restrictions and existing binary scalar functions. Reject unsupported
collection kernels before coercion.
- Preserve historical second offsets in both TIMESTAMPTZ formatting and
parsing. Skip masked NULL payloads, reject unrepresentable local years, and
preserve cast error/NULL behavior at boundaries.
Arrow convertor migration, Parquet/ORC semantics, external writer changes
and catalog mapping migration belong to the subsequent extractions. This PR
does not enable native VARBINARY storage.
### Master adaptation
- Retain the fixed-offset normalization and tests already present on master.
- Use the current void-returning `VInPredicate::_prepare_zonemap_min_max`
interface in both the guard and its test.
- Retain master header cleanup and existing timestamp-nanosecond tests.
- Retain the existing master binary-literal encoder and its StringView input
contract; the older std::string-based caller fix is not applicable.
### Testing
- BE ASAN build and **199 tests passed** across 17 suites using
`run-be-ut.sh`, including binary lifetime/SerDe/rejection, timestamp
parsing/casts, and existing Arrow/Variant serialization coverage.
- `VarBinaryUnsupportedCollectionTest`: **passed** (13 unsupported
expressions plus supported byte-preserving collection analysis). The FE test
reactor and repository Checkstyle passed after cleaning stale branch build
artifacts.
- Repository clang-format 16 check and build-header hygiene checks:
**passed**; 31 changed C++ source/header files.
- Groovy compilation of the three regression suites: **passed**. Live SQL
regression execution remains pending CI.
- clang-tidy was attempted but could not complete because master already
contains an unmatched `NOLINTEND` in `be/src/core/types.h`. A diagnostic run
with the compiler resource directory corrected reproduced that blocker; the
other reported findings in `column_varbinary.cpp` were outside changed lines.
This is not a clean clang-tidy result.
The focused BE test source list and local test/build settings were restored
before committing. No build configuration changes are included.
### Release note
Fix binary value lifetime and serialization, reject unsupported binary
computation paths, and preserve TIMESTAMPTZ historical offsets and boundary
error behavior.
### Check List (For Author)
- Test
- [x] Regression test (three self-checking suites added; execution pending
CI)
- [x] Unit Test
- Behavior changed:
- [x] Yes. Binary rejection and timestamp boundary behavior are described
above.
- Does this need documentation?
- [x] No. This fixes existing type behavior without introducing a
configuration option.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
