Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2025-02-09 Thread via GitHub
github-actions[bot] commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2646674179 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2025-02-09 Thread via GitHub
github-actions[bot] closed pull request #11612: Spark 3.5: Refactor scanning changelog table with timestamps URL: https://github.com/apache/iceberg/pull/11612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2025-02-02 Thread via GitHub
github-actions[bot] commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2629642276 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2025-01-02 Thread via GitHub
flyrain commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2568621811 > 2. `endSnapshot` is `null` (`endTimestamp == null` ensures this is from calculation) Makes sense to check 2 within `if (startTimestamp != null || endTimestamp != null)` with th

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2025-01-02 Thread via GitHub
manuzhang commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2568552990 @flyrain Thanks for taking a look. I think what might be confusing is `startSnapshotId` and `endSnapshotId` could be passed in or calculated from `startTimestamp` and `endTimestamp`.

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2025-01-02 Thread via GitHub
flyrain commented on code in PR #11612: URL: https://github.com/apache/iceberg/pull/11612#discussion_r1901376613 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -560,20 +560,15 @@ public Scan buildChangelogScan() { } boo

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2024-12-12 Thread via GitHub
manuzhang closed pull request #11612: Spark 3.5: Refactor scanning changelog table with timestamps URL: https://github.com/apache/iceberg/pull/11612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2024-11-25 Thread via GitHub
manuzhang commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2499346791 @flyrain please help review this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2024-11-21 Thread via GitHub
manuzhang commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2491394278 @Acehaidrey When end snapshot is set with `endTimestamp` but is null, it means no valid end snapshot can't be found and hence the scan is empty. -- This is an automated message from

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2024-11-21 Thread via GitHub
Acehaidrey commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2491106772 Hi Manu can you explain to me why the end snapshot matters here , sorry I may have missed your point (new to the codebase as well so getting familiar) -- This is an automated messa

Re: [PR] Spark 3.5: Refactor scanning changelog table with timestamps [iceberg]

2024-11-21 Thread via GitHub
manuzhang commented on PR #11612: URL: https://github.com/apache/iceberg/pull/11612#issuecomment-2490330889 Submit https://github.com/apache/iceberg/pull/11616 to fix test failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH