OIiveirra opened a new pull request, #69032:
URL: https://github.com/apache/doris/pull/69032
### What problem does this PR solve?
Issue Number: N/A
Related PR: #67907
Problem Summary: Add Iceberg V3 GEOMETRY/GEOGRAPHY external-table support on
master using the current FE type and Connector SPI interfaces. Preserve kind,
CRS and Geography algorithm across schema mapping, block transport and raw XY
WKB Parquet reads/writes. Validate DDL and INSERT/UPDATE/MERGE identity before
casts, including branch projections and CTEs.
## Summary
```text
Iceberg V3 schema -> FE / Connector SPI -> parameterized spatial columns
-> raw XY WKB Parquet -> writeback -> fresh catalog read
```
Support all seven basic XY WKB kinds, EMPTY and nullable values. Raw
accessors preserve bytes and projected coordinates. Nondefault Geography
metadata can be stored and written back; measurement/relationship functions are
limited to supported shapes with OGC:CRS84 + spherical. Internal tables and
nested spatial writes are rejected. Spatial storage support is limited to
Parquet.
Historical ST_GeometryFromWKB/ST_GeomFromWKB keep STRING/EWKB behavior;
ST_GeometryFromWKBTyped is the explicit typed Geometry constructor. Generic
Flight clients must project ST_AsBinary/ST_AsText instead of returning spatial
columns directly.
This is a master implementation associated with the earlier branch-4.1 PR;
it does not change or rewrite that PR. Spatial indexes, bounds pruning,
predicate pushdown, Paimon, Lance, GeoArrow, other Geography computation
algorithms, CRS conversion and Z/M/typed EWKB SRID are outside scope.
## Evidence
- **Before:** Physical binary-to-spatial reads, parameterized block
transport and hashing had missing execution paths. UPDATE failed before spatial
validation; MERGE target casts could hide mismatched source CRS.
**After:** Targeted BE tests cover these paths. Actual SQL rejects
kind/CRS/algorithm mismatches for INSERT, UPDATE and direct/subquery/CTE MERGE,
while matching and NULL writes succeed.
- **Before:** External regression had no complete generated result and
fresh-catalog proof.
**After:** Standard Iceberg spatial suite generated results and passed
without generation. All seven kinds have nonempty coverage, with EMPTY/partial
NaN, remote nested reads, grouping and fresh-catalog byte/metadata checks.
Existing GIS 4/4 suites pass against unchanged golden files.
- FE targeted type/schema/write/signature tests and complete Connector SPI
147/147 pass; related final sink tests 37/37 and MERGE tests 30/30 pass. BE
spatial/execution tests 184/184 pass, followed by 6/6 tests after test-helper
refactoring.
- Standard FE/BE builds pass. BE uses the personal ASAN/clang21 profile with
GLIBC_COMPATIBILITY=OFF; this is not a default CI or RELEASE ABI/performance
claim. FE Checkstyle, clang-format16 and header hygiene pass. Feature
clang-tidy covers all 40 changed BE files; its two test-complexity errors were
fixed and that file passed full-file recheck.
## Merge Danger
**Door:** Two-way for code; newly written Iceberg V3 spatial schema/data
requires compatible readers.
**Blast Radius:** External-table types and shared type transport. Protocol
fields are optional and ids were audited against maintained release branches.
No internal-table storage or automatic coordinate conversion is introduced.
### Release note
Support Iceberg V3 GEOMETRY and GEOGRAPHY external-table schema mapping and
raw XY WKB Parquet read/write with parameter preservation and explicit
computation limits.
### Check List (For Author)
- Test
- [x] Regression test: nonConcurrent test_iceberg_spatial_v3, generated
then verified without genOut; existing spatial_functions P0 directory 4/4.
- [x] Unit Test: targeted FE/BE and complete Connector SPI as detailed
above.
- [x] Manual test: six documented typed-constructor examples on the
isolated task cluster.
- Behavior changed:
- [x] Yes. Iceberg V3 spatial columns and typed constructors are
available; unsupported writes/computations fail explicitly.
- Does this need documentation?
- [x] Yes. https://github.com/apache/doris-website/pull/4239
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
