mxm opened a new pull request, #17038:
URL: https://github.com/apache/iceberg/pull/17038
This PR fixes the CI test flakiness seen in:
```
TestConvertEqualityDeletesE2E > testConvertEqualityDeletesE2E(String) > [1]
staging FAILED
org.opentest4j.AssertionFailedError:
expected: 2L
but was: 1L
at
app//org.apache.iceberg.flink.maintenance.api.TestConvertEqualityDeletesE2E.lambda$testConvertEqualityDeletesE2E$1(TestConvertEqualityDeletesE2E.java:127)
```
e.g.:
https://github.com/apache/iceberg/actions/runs/28499724055/job/84473933155?pr=16293
Flink's table maintenance framework maintains a lock to prevent concurrent
execution of maintenance tasks. The component responsible for removing the lock
(LockRemover) releases the task lock once a watermark reaches it past the
task's start timestamp.
EqualityConvertPlanner emits phase watermarks in the middle of its
execution, and the committer forwarded them, so the lock was released before
the run completed. The maintenance framework then started a next cycle
concurrently, and both re-processed the same uncommitted staging snapshot. The
overlapping commits conflicted, and one advanced its commit marker while
dropping its deletion vector, causing the test flakiness.
The solution is to forward watermarks from the committer only after it
finishes the conversion cycle, to ensure mutual exclusive execution of the
maintenance tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
