Anurag-S1ngh opened a new pull request, #4406: URL: https://github.com/apache/iceberg-python/pull/4406
## Summary Filtered scans raised or returned wrong rows at two kinds of edge values. - Year and month partitions raised `OverflowError` when a filter projected past 9999-12-31 or before 0001-01-01, because the transform converted the day count to a `datetime.date`. 9999-12-31 is a common "no end date" value. - `EqualTo`, `In`, and `NotEqualTo` on `inf` returned wrong rows for float columns. A Python float becomes a `DoubleLiteral`, and converting that to a float column turned `inf` into `FloatAboveMax` or `FloatBelowMin`, which bound the predicate to `AlwaysFalse` or `AlwaysTrue`. ## Changes - Compute years and months from a day count with integer arithmetic, so a projection just past the last representable date yields year 10000 and matches no partition instead of raising. Results inside the representable range are unchanged. - Convert `inf` and `-inf` to a `FloatLiteral` for float columns, like any other float that fits. ## Testing - Added unit tests for the day, microsecond, and nanosecond conversions at and beyond the date range, the year projection at both date limits, and the infinity literal conversion. - Added scan tests for `inf`/In filters on a float column and a year-partitioned table scanned at 9999-12-31. - `make test` passes (4703 passed, 5 skipped). Closes #4404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
