mxm commented on code in PR #17630:
URL: https://github.com/apache/iceberg/pull/17630#discussion_r3870157439
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertPlanner.java:
##########
@@ -267,52 +293,79 @@ public void processElement(StreamRecord<Trigger> element)
throws Exception {
}
/**
- * Brings the worker's index up to date with the current state of the target
branch:
- *
- * <ul>
- * <li>Updates {@link #lastStagingSnapshotId} from the most recent
committer marker on main.
- * <li>Bootstraps the index from main on the first trigger with a non-null
main snapshot.
- * <li>Reindexes from main when external commits (e.g. compaction or
direct writes) have
- * advanced main past the currently-indexed snapshot.
- * </ul>
- *
- * <p>No-op when main hasn't moved since the last trigger. Otherwise the
history walk is bounded
- * to commits added since {@link #lastMainSnapshotId}.
+ * Updates {@link #lastStagingSnapshotId} from the most recent committer
marker on the target
+ * branch. Returns the discovered work, or null when the target has not
moved since the last
+ * trigger and the cursor therefore cannot have changed.
*/
- private void ensureIndexCurrent(Snapshot mainSnapshot) {
+ private LastCommittedWork refreshStagingCursor(Snapshot mainSnapshot) {
Long currentMainSnapshotId = mainSnapshot != null ?
mainSnapshot.snapshotId() : null;
if (Objects.equals(lastMainSnapshotId, currentMainSnapshotId)) {
- return;
+ return null;
}
LastCommittedWork info = discoverLastCommittedWork(mainSnapshot);
updateLastStagingSnapshotId(info);
+ lastMainSnapshotId = currentMainSnapshotId;
Review Comment:
I'm not sure we can advance the pointer here already because we haven't
updated the index yet. I think this needs to go back into `ensureIndexCurrent`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
