kevinjqliu commented on code in PR #1206:
URL: https://github.com/apache/iceberg-python/pull/1206#discussion_r1775434612
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1068,20 +1068,13 @@ def primitive(self, primitive: pa.DataType) ->
PrimitiveType:
return StringType()
elif pa.types.is_date32(primitive):
return DateType()
- elif isinstance(primitive, pa.Time64Type) and primitive.unit == "us":
+ elif isinstance(primitive, pa.Time64Type) and primitive.unit in ["us",
"ns"]:
return TimeType()
elif pa.types.is_timestamp(primitive):
primitive = cast(pa.TimestampType, primitive)
- if primitive.unit in ("s", "ms", "us"):
+ if primitive.unit in ("s", "ms", "us", "ns"):
# Supported types, will be upcast automatically to 'us'
pass
- elif primitive.unit == "ns":
- if self._downcast_ns_timestamp_to_us:
- logger.warning("Iceberg does not yet support 'ns'
timestamp precision. Downcasting to 'us'.")
- else:
- raise TypeError(
- "Iceberg does not yet support 'ns' timestamp
precision. Use 'downcast-ns-timestamp-to-us-on-write' configuration property to
automatically downcast 'ns' to 'us' on write."
- )
Review Comment:
this is still necessary because iceberg doesnt support `ns`. this workaround
help us downcast a column with `ns` to `us`
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1068,20 +1068,13 @@ def primitive(self, primitive: pa.DataType) ->
PrimitiveType:
return StringType()
elif pa.types.is_date32(primitive):
return DateType()
- elif isinstance(primitive, pa.Time64Type) and primitive.unit == "us":
+ elif isinstance(primitive, pa.Time64Type) and primitive.unit in ["us",
"ns"]:
Review Comment:
dont need to check for precision here, since its `us` and `ns` are supported.
Although we need to add code to downcast `ns` to `us`, similar to below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
