Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-05-02 Thread via GitHub
corleyma commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2092076360 > More over, multiple different snapshots can also be committed between two consecutive metadata json files. In what situations would that occur? In my (possibly incorrec

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-05-02 Thread via GitHub
syun64 commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2091529571 > * the path to the metadata json file for a given snapshot id. > * I really wish this was a property of the Snapshot class; is that possible or does this break correspondenc

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-04-19 Thread via GitHub
gupteaj commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2067221356 Presto time travel reference - https://prestodb.io/docs/0.286/connector/iceberg.html#time-travel-using-version-system-version-and-timestamp-system-time Time travel for snapsho

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-04-17 Thread via GitHub
kevinjqliu commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2062828600 > an api like Table.as_of(snapshot_id/timestamp) -> Snapshot would be useful Yea, it's helpful in situations where we need to manipulate the Table state and get back th

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-04-17 Thread via GitHub
corleyma commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2062571021 Still, an api like `Table.as_of(snapshot_id/timestamp) -> Snapshot` would be useful, even if reading requires then passing the correct arguments to `Table.scan`. In general it

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-04-17 Thread via GitHub
kevinjqliu commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2062244023 +1, I agree with you. Passing the snapshot-id should be an engine-specific implementation detail. I was thinking about the Spark/Trino syntax of `AS OF / `. I interpr

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

2024-04-13 Thread via GitHub
syun64 commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2053740059 I think this is a great discussion item @kevinjqliu - thank you for raising this. I'm a bit torn between whether we (PyIceberg) should be responsible for creating separat