The GitHub Actions job "Flink CI (beta)" on flink.git/master has succeeded.
Run started by GitHub user pnowojski (triggered by pnowojski).

Head commit for run:
a8835bd37312af00d6669d3ebe4caca70cf3f220 / Efrat Levitan 
<[email protected]>
[FLINK-40093][Runtime] Resume idle splits by alignment check if previously 
paused

Usually pauseOrResumeSplits pauses idleness timer for the split so it isn't 
marked idle while paused. However with low idleness timeout (observed with 1s) 
+ low allowed WM drift, a race condition could cause paused splits to never 
resume though they have records:

1. A split becomes paused due to too advanced records.
2. pauseOrResumeSplits pauses the split.
3. pauseOrResumeSplits reaches to pause the split idleness clock but is 
{idlenessTimeout} too late, and the split becomes idle.
4. The watermark advances but the split is excluded from the watermark 
alignment check due to its idleness.
5. More records arrive but the split is paused at the connector level so they 
aren't processed, nor seen by watermarkGenerator so it still considers the 
split idle
The PR aims to fix it by preserving the part where idle splits are excluded 
from alignment pause (to not override their idle status) while allowing 
alignment check to resume splits even if they are currently idle.
They are considered idle until they emit the next record.

[FLINK-40093][Runtime] Refactor the test to reproduce the race condition

Report URL: https://github.com/apache/flink/actions/runs/29567928604

With regards,
GitHub Actions via GitBox

Reply via email to