Fokko commented on code in PR #2173:
URL: https://github.com/apache/iceberg-python/pull/2173#discussion_r2215011985
##########
pyiceberg/utils/schema_conversion.py:
##########
@@ -69,8 +69,10 @@
LOGICAL_FIELD_TYPE_MAPPING: Dict[Tuple[str, str], PrimitiveType] = {
("date", "int"): DateType(),
("time-micros", "long"): TimeType(),
+ ("timestamp-millis", "int"): TimestampType(),
Review Comment:
@matthias-Q To make that work, we would need to add another type as well,
but that would suggest that milliseconds are part of the Iceberg spec, and that
would require more discussion on the specification level.
The code that you're referencing converts an Iceberg schema to an Arrow
schema. An Iceberg schema does not allow for millisecond resolution timestamps,
so that's not the right place to fix this.
You can think of this situation as a column promotion where the type has
been widened from `millis` to `micros`, but since we convert pretty early from
Avro to Iceberg, we also lose that information:
https://github.com/apache/iceberg-python/blob/ad8263b1be048c8cb67d40efe70f494a4f1cb374/pyiceberg/avro/resolver.py#L448-L455
Let me think about it, but it doesn't seem to be trivial.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
