huan233usc opened a new pull request, #17161:
URL: https://github.com/apache/iceberg/pull/17161
Computes a bounding box for `geometry` columns in Parquet so they can carry
`lower_bounds`/`upper_bounds` and participate in data skipping. This is the
first
slice of the geo bounds work (Phase 2), scoped to the clean, unambiguous
case.
## Problem
The Parquet footer's lexicographic min/max over WKB bytes is not meaningful
for
spatial values, so `ParquetMetrics` diverts geometry/geography columns to
counts-only metrics. As a result geometry columns get no bounds and cannot be
data-skipped.
## Approach
Compute the box **at write time**, since the footer cannot provide it. This
reuses
the existing writer-side value-scanning metrics channel — the same path
`float`
and `double` already use to track NaN counts the footer can't give
(`ParquetValueWriter.metrics()` → `ParquetWriter.metrics()`):
- A new `GeometryWriter` writes byte-identical WKB **and** folds each
value's XY
coordinates into a running box, emitting a `FieldMetrics<GeospatialBound>`
whose
lower/upper corners are the box. These serialize into the existing
`lower_bounds`/`upper_bounds` maps through the geometry `Conversions`
case. No
changes to `ParquetMetrics`, `ParquetWriter`, or `Conversions`.
- The value-scanned metrics take precedence over the counts-only footer
branch, and
the optional-field writer reconciles the null count, so the builder tracks
only
non-null values.
`WKBBoundingBox` (new, in `api/geospatial` next to
`GeospatialBound`/`BoundingBox`)
is a pure-Java WKB coordinate scanner — **no JTS dependency**. It walks all
OGC
geometry types (point, linestring, polygon, multi*, collection), handles
both byte
orders (per-geometry, so mixed-endian collections work), reads past Z/M
(XY-only
box), skips NaN coordinates (so `POINT EMPTY` contributes nothing, per
spec), and
validates the buffer defensively (truncation, bad byte order, unknown type,
and
oversized counts all raise `IllegalArgumentException` rather than reading
out of
bounds).
## Scope
**GEOMETRY only, 2D (XY), planar** — the case the spec calls straightforward
and
unambiguous. Deliberately left as follow-ups:
- **GEOGRAPHY** bounds (longitude periodicity, latitude extrema via edges,
numeric
correction, out-of-range coordinate policy) — needs its own numeric design.
- Higher dimensions (XYZM), the `content_stats` `geo_lower`/`geo_upper`
struct write
bridge, ORC/Avro geo bounds, and CRS validation.
## Tests
- `TestWKBBoundingBox` — the scanner across all geometry types, Z/M skip,
both/mixed
endianness, nested collections, empties, NaN skip, ±Inf, and malformed
inputs.
- `TestGeometryFieldMetrics` — the builder's bounds, counts, and
empty/no-value cases.
- `TestMetrics.testMetricsForGeospatialTypes` (new, exercised via
`TestParquetMetrics`)
— end-to-end: two geometry points produce the expected bbox in
`lower_bounds`/
`upper_bounds`; geography stays bounds-less; ORC is skipped (geo is
Parquet-only).
- `TestParquetDataWriter.testGeospatialRoundTrip` still passes unchanged,
confirming
the WKB bytes written are byte-identical.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
