anoopj opened a new issue, #3503:
URL: https://github.com/apache/iceberg-rust/issues/3503
Follow-up to #3483. The spec lists date -> timestamp and date ->
timestamp_ns as valid v3+ primitive promotions (timestamptz/timestamptz_ns are
explicitly not allowed). As with int -> long, the Avro manifest format does not
store the type of lower/upper bounds and promotion does not rewrite them, so a
value written under date must be decoded and compared using its written type.
iceberg-rust only handles int -> long and float -> double:
Even though this is a latent bug, users don't hit this because iceberg-rust
cannot produce this promotion itself today (no column-type update in core), so
no user hits it via a round trip. But a date column promoted to timestamp by a
v3-capable engine is a valid table that iceberg-rust reads, and such a read
would compare date-days against timestamp-micros: wrong pruning and, for a
promoted partition source, detached deletes, pretty much the same class of
silent mismatch #3483 fixes for int/float.
## Possible fix
Teach the value/bound promotion path about the date promotions, ideally as
part of moving bound promotion into the reader (the sibling follow-up), so
there is one promotion point
## Tests
bound pruning where a promoted date column's bounds decide a file;
equality-delete / deletion-vector matching across a date -> timestamp
promotion; and an overflow case for timestamp_ns asserting a runtime error
rather than a wrapped value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
