syun64 commented on code in PR #910:
URL: https://github.com/apache/iceberg-python/pull/910#discussion_r1674689843
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1320,7 +1321,16 @@ def _cast_if_needed(self, field: NestedField, values:
pa.Array) -> pa.Array:
and pa.types.is_timestamp(values.type)
and values.type.unit == "ns"
):
- return values.cast(target_type, safe=False)
+ if target_type.tz == "UTC" and values.type.tz in
UTC_ALIASES or not target_type.tz and not values.type.tz:
+ return values.cast(target_type, safe=False)
+ if (
+ pa.types.is_timestamp(target_type)
+ and target_type.unit == "us"
+ and pa.types.is_timestamp(values.type)
+ and values.type.unit in {"s", "ms", "us"}
Review Comment:
Sorry this is a bit confusing, I agree 😓 . I included it here because I
wanted to support casting `pa.timestamp('us', tz='Etc/UTC')` to
`pa.timestamp('us', tz='UTC')` within the same condition.
I think we won't hit this condition if both the input and requested types
are `pa.timestamp('us')` because we enter this block only if target_type and
values.type are not equal:
https://github.com/apache/iceberg-python/blob/main/pyiceberg/io/pyarrow.py#L1315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
