huan233usc opened a new pull request, #17119:
URL: https://github.com/apache/iceberg/pull/17119
## Summary
`TypeToSchema` mapped every Iceberg primitive to an Avro schema except
`geometry` and `geography`,
so converting a schema with a geo column threw
`UnsupportedOperationException: Unsupported type ID:
GEOMETRY` before any value could be read or written through the Avro object
model. This wires up the
value path.
Per the Avro type mapping in the spec, `geometry` and `geography` are stored
as an Avro `bytes` field
carrying WKB — the same representation as `binary`:
| Iceberg type | Avro type |
| --- | --- |
| `binary` | `bytes` |
| `geometry` | `bytes` (WKB) |
| `geography` | `bytes` (WKB) |
So `TypeToSchema.primitive()` now maps both geo types to the existing
`BINARY_SCHEMA`. The value
read/write paths dispatch on the Avro physical type, so once the schema is
`bytes` the existing
`byteBuffers` reader/writer handle geo unchanged in both the generic and
internal object models (geo
values are WKB `ByteBuffer`s, exactly like `binary`) — no value-path code
changes are needed.
This is a follow-up split out of #16982 (which added the Parquet WKB value
path and deliberately left
the Avro object model as a follow-up).
### Note on the reverse direction
`SchemaToType` intentionally still maps Avro `bytes → binary`. Plain Avro
`bytes` carries no marker to
distinguish geometry/geography from binary, and Iceberg resolves the real
column type from the
expected/table schema, never by reverse-inferring from the file's Avro
schema — the same way `binary`
and other `bytes`-backed types already behave.
## Test plan
- Enable the shared `DataTest` geospatial coverage (`supportsGeospatial()`)
for the Avro object models:
`TestGenericAvro`, `TestInternalAvro`, and the generic-data
`TestGenericData`. This round-trips
geometry and geography across multiple CRS and edge algorithms with
randomly generated WKB values.
- Add the `GEOMETRY`/`GEOGRAPHY` → `ByteBuffer` handling that the generic
Avro path needs in
`RandomAvroData` and `AvroTestHelpers`, mirroring the existing
`RandomInternalData` /
`InternalTestHelpers` handling.
- `./gradlew :iceberg-core:test --tests
org.apache.iceberg.avro.TestGenericAvro --tests
org.apache.iceberg.avro.TestInternalAvro`
- `./gradlew :iceberg-data:test --tests
org.apache.iceberg.data.avro.TestGenericData`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
