JFinis opened a new issue, #9713: URL: https://github.com/apache/iceberg/issues/9713
### Apache Iceberg version 1.4.3 (latest release) ### Query engine Spark ### Please describe the bug 🐞 The hour partition transform returns an `int`. For very large dates, this function will overflow. Consequently this partition transform is not order preserving, resulting in incorrect partition pruning. Consider the following Spark SQL: ``` CREATE TABLE t (x timestamp) PARTITIONED BY(hour(x)) TBLPROPERTIES(`format-version`=2); INSERT INTO iceberg.sample VALUES(timestamp '250000-01-01 00:00:00'); SELECT * FROM iceberg.sample WHERE x > timestamp '2020-01-01 00:00:00'; ``` The last query will incorrect return an empty result, as the `hour(timestamp '250000-01-01 00:00:00')` is `-2120780968`, so the partition for this row incorrectly is a very low date. Thus, the partition pruning will incorrectly prune this partition in the query, as it considers it smaller than a 2020 date. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
