sohailshk opened a new pull request, #4361:
URL: https://github.com/apache/iceberg-python/pull/4361

   # Rationale for this change
   
   `Table.snapshot_as_of_timestamp` walks the snapshot log backwards and 
returns the first entry at or before the timestamp, which assumes the log is 
ordered by timestamp. The spec does not guarantee that, and Java's 
TableMetadata accepts a log entry up to one minute older than the previous one, 
so inside that skew window the lookup can return an older snapshot. Java fixed 
the same assumption in `SnapshotUtil.nullableSnapshotIdAsOfTime` 
(apache/iceberg#17360, "finding the entry with the maximum timestamp <= 
requested timestamp").
   
   This picks the entry with the largest timestamp that is at or before (or 
strictly before, with `inclusive=False`) the requested one. On equal timestamps 
the later entry wins, unlike Java which keeps the first: PyIceberg logs every 
snapshot added in one transaction with its own timestamp_ms, so two can share a 
millisecond and the later one is the state that was actually current. 
Later-wins is also what PyIceberg and Java did before. 
`inspect.metadata_log_entries` uses the same lookup for `latest_snapshot_id`.
   
   ## Are these changes tested?
   
   Yes, `test_snapshot_by_timestamp_with_unordered_snapshot_log` in 
tests/table/test_init.py (two entries 10 seconds out of order, plus a tie; 
fails on main, passes here). tests/table/test_init.py and 
tests/table/test_inspect.py pass.
   
   ## Are there any user-facing changes?
   
   Time travel by timestamp returns the right snapshot when the snapshot log 
has entries slightly out of timestamp order.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to