sargarass opened a new issue, #13679:
URL: https://github.com/apache/iceberg/issues/13679
### Apache Iceberg version
None
### Query engine
None
### Please describe the bug 🐞
When PyIceberg encounters a table column of type Timestamp where the Parquet
file's logical type is nanoseconds, it downcasts it to microseconds.
```
Iceberg does not yet support 'ns' timestamp precision. Downcasting to 'us'.
Iceberg does not yet support 'ns' timestamp precision. Downcasting to 'us'.
pyarrow.Table
event_time: timestamp[us, tz=UTC] not null
insertion_no: int64 not null
exchange_time: timestamp[us, tz=UTC] not null
sequence_no: int64 not null
side: int8 not null
price: double not null
amount: double not null
is_eot: bool not null
_feed: int8 not null
_feed_l2_depth: int32 not null
----
event_time: [[2025-07-24 00:00:00.006136Z,2025-07-24
00:00:00.006136Z,2025-07-24 00:00:00.006136Z]]
insertion_no: [[8298661145,8298661146,8298661147]]
exchange_time: [[2025-07-24 00:00:00.003000Z,2025-07-24
00:00:00.003000Z,2025-07-24 00:00:00.003000Z]]
sequence_no: [[8129311262230,8129311262230,8129311262230]]
side: [[1,1,1]]
price: [[3627.72,3627.77,3627.95]]
amount: [[1.88,1.409,4.17]]
is_eot: [[false,false,false]]
_feed: [[0,0,0]]
_feed_l2_depth: [[0,0,0]]
```
In contrast, Spark ignores this entirely, resulting in Spark timestamps
appearing far in the future.
```
spark-sql ()> select * from nyc.with_logic_type22 limit 3;
+57530-05-08 03:00:06.136643 8298661145 +57530-05-08 03:00:03
8129311262230 1 3627.72 1.88 false 0 0
+57530-05-08 03:00:06.136643 8298661146 +57530-05-08 03:00:03
8129311262230 1 3627.77 1.409 false 0 0
+57530-05-08 03:00:06.136643 8298661147 +57530-05-08 03:00:03
8129311262230 1 3627.95 4.17 false 0 0
````
Actual data look like this:
```
./clickhouse client -q "SELECT * FROM
file('2025-07-24T00-00-00_2025-07-25T00-00-00_01K11MWTJ1RX6Q7VRZP7YSN490.parquet',
Parquet) order by (event_time, insertion_no) LIMIT 3;" --port 9002
2025-07-24 00:00:00.006136643 8298661145 2025-07-24
00:00:00.003000000 8129311262230 1 3627.72 1.88 false 0 0
2025-07-24 00:00:00.006136643 8298661146 2025-07-24
00:00:00.003000000 8129311262230 1 3627.77 1.409 false 0 0
2025-07-24 00:00:00.006136643 8298661147 2025-07-24
00:00:00.003000000 8129311262230 1 3627.95 4.17 false 0 0
```
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
