Anurag-S1ngh opened a new pull request, #4406:
URL: https://github.com/apache/iceberg-python/pull/4406

   ## Summary
   
   Filtered scans raised or returned wrong rows at two kinds of edge values.
   
   - Year and month partitions raised `OverflowError` when a filter projected 
past 9999-12-31 or before 0001-01-01, because the transform converted the day 
count to a `datetime.date`. 9999-12-31 is a common "no end date" value.
   - `EqualTo`, `In`, and `NotEqualTo` on `inf` returned wrong rows for float 
columns. A Python float becomes a `DoubleLiteral`, and converting that to a 
float column turned `inf` into `FloatAboveMax` or `FloatBelowMin`, which bound 
the predicate to `AlwaysFalse` or `AlwaysTrue`.
   
   ## Changes
   
   - Compute years and months from a day count with integer arithmetic, so a 
projection just past the last representable date yields year 10000 and matches 
no partition instead of raising. Results inside the representable range are 
unchanged.
   - Convert `inf` and `-inf` to a `FloatLiteral` for float columns, like any 
other float that fits.
   
   ## Testing
   
   - Added unit tests for the day, microsecond, and nanosecond conversions at 
and beyond the date range, the year projection at both date limits, and the 
infinity literal conversion.
   - Added scan tests for `inf`/In filters on a float column and a 
year-partitioned table scanned at 9999-12-31.
   - `make test` passes (4703 passed, 5 skipped).
   
   Closes #4404


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to