flyrain commented on code in PR #6350:
URL: https://github.com/apache/iceberg/pull/6350#discussion_r1044897305
##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java:
##########
@@ -285,6 +286,36 @@ public Scan buildChangelogScan() {
Long startSnapshotId = readConf.startSnapshotId();
Long endSnapshotId = readConf.endSnapshotId();
+ Long startTimestamp = readConf.startTimestamp();
+ Long endTimestamp = readConf.endTimestamp();
+
+ Preconditions.checkArgument(
+ !(startSnapshotId != null && startTimestamp != null),
+ "Cannot set neither %s nor %s for changelogs",
+ SparkReadOptions.START_SNAPSHOT_ID,
+ SparkReadOptions.START_TIMESTAMP);
+
+ Preconditions.checkArgument(
+ !(endSnapshotId != null && endTimestamp != null),
+ "Cannot set neither %s nor %s for changelogs",
+ SparkReadOptions.END_SNAPSHOT_ID,
+ SparkReadOptions.END_TIMESTAMP);
+
+ if (startTimestamp != null && endTimestamp != null) {
+ Preconditions.checkArgument(
+ startTimestamp < endTimestamp,
+ "Cannot set %s to be greater than %s for changelogs",
+ SparkReadOptions.START_TIMESTAMP,
+ SparkReadOptions.END_TIMESTAMP);
+ }
+
+ if (startTimestamp != null) {
+ startSnapshotId = getStartSnapshotId(startTimestamp);
+ }
+
+ if (endTimestamp != null) {
+ endSnapshotId = SnapshotUtil.snapshotIdAsOfTime(table, endTimestamp);
+ }
Review Comment:
Good point! Added. Although, we could support time range in incremental as
well in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
