The GitHub Actions job "Nightly (beta)" on flink.git/release-1.20 has failed. Run started by GitHub user github-actions[bot] (triggered by github-actions[bot]).
Head commit for run: a578073b8c1c5d48acc3f2985e6b28fe57fa5fe0 / Martijn Visser <[email protected]> [FLINK-40475][runtime] Fix watermark loss and stall in StatusWatermarkValve when subpartitions realign after idleness (#29024) [FLINK-40475][runtime] Fix watermark loss and stall in StatusWatermarkValve when subpartitions realign after idleness The emission logic of StatusWatermarkValve relies on two invariants over the set of watermark-aligned subpartitions: it is only empty when no subpartition is active, and whenever it is non-empty its min watermark equals the last output watermark. Both were violated once an unaligned subpartition - one that went idle, resumed, and only partially caught up to the last output watermark - was involved: - The FLINK-7728 all-idle flush was skipped unless the last subpartition to become idle held the current min watermark. That assumption doesn't hold when unaligned subpartitions are involved, since an unaligned subpartition's watermark is invisible to the min-derive path. This made the final watermark depend on the order in which inputs became idle - exactly the defect FLINK-7728 was meant to fix. - The idle->active branch re-added a caught-up subpartition to the aligned set without re-deriving the min. When no other aligned subpartition remained, the realigned subpartition's watermark stalled indefinitely, emitted only once an even larger watermark arrived on it. Fix both by re-deriving from the aligned set unconditionally instead of relying on the broken shortcut: - Flush the max watermark unconditionally once all subpartitions go idle. findAndOutputMaxWatermarkAcrossAllSubpartitions only emits when the max advances past the last output watermark, so this cannot regress monotonicity, and it keeps the valve consistent with CombinedWatermarkStatus, which has flushed unconditionally since FLINK-38454. - Re-derive (and possibly emit) the min watermark after a subpartition realigns, after the ACTIVE status is emitted so downstream inputs don't drop the watermark while they still consider the input idle. Documents the invariants on alignedSubpartitionStatuses. Generated-by: Claude Code (claude-fable-5) (cherry picked from commit bb773b5fb8a4a87dd4efc3a8ae261b7e08e602a6) (cherry picked from commit 022b7dce9385b5b40bf0e5169f2171c767232880) Report URL: https://github.com/apache/flink/actions/runs/42739759214 With regards, GitHub Actions via GitBox
