huan233usc commented on PR #17509:
URL: https://github.com/apache/iceberg/pull/17509#issuecomment-5208904560
Thanks — replies to two of the questions that weren't inline.
**Parquet already computes a bounding box.** Its footer box doesn't follow
the Iceberg spec. `format/spec.md:772` skips NaN per coordinate, so a value
with a valid ordinate in one dimension and NaN in the other still contributes
the valid one. Parquet's `BoundingBox.update(Geometry)` takes the JTS envelope,
and JTS `expandToInclude` has no NaN guard, so one NaN vertex makes the
envelope's Y range NaN; the `updateBounds` guard then drops that dimension
entirely:
```
MULTIPOINT((1 NaN),(5 2))
Parquet footer -> x=[1,5] y=[Infinity,-Infinity] (Y lost)
spec / this PR -> x=[1,5] y=[2,2]
```
Iceberg can't recover the dropped dimension from the footer value
afterwards. The Avro path in #17119 has no footer statistics to read at all, so
a collector is needed regardless. Will coordinate with #12347.
**Should invalid-but-parseable WKB fail the write?** The line I'd draw is
spec legality, not parseability. Z/M is legal per `format/spec.md:2138` ("ISO
supporting XY, XYZ, XYM, XYZM"), so it now contributes bounds rather than
failing. A `LINESTRING` child inside a `MULTIPOINT` violates OGC — same class
as truncation, trailing bytes and a bad byte-order flag, all already rejected
here. Accepting malformed WKB silently would produce bounds that are quietly
wrong and prune files that should match, which seems worse than failing the
write.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
