The GitHub Actions job "Flink CDC CI" on flink-cdc.git/FLINK-39775-release-snapshot-split-metadata has failed. Run started by GitHub user spoorthibasu (triggered by lvyanquan).
Head commit for run: 78e0b7f72058fcb1bddebf33ad8234a950d4df7d / Spoorthi Basu <[email protected]> [FLINK-39775][mysql] Release snapshot split metadata after entering the stream phase After the snapshot phase finishes and the binlog split is assigned, the source coordinator kept the full snapshot split metadata (assigned splits, finished offsets, table schemas) around forever. snapshotState() rebuilt the whole SnapshotPendingSplitsState on every checkpoint, notifyCheckpointComplete() never dropped it, and the enumerator held a second copy in its binlog meta cache. For a table with many finished splits (around 300K for a 2.5B-row table at the default chunk size) this kept JobManager memory high and pushed up direct memory during checkpoint serialization, eventually OOM-killing the container. The reader now tells the coordinator (BinlogSplitMetaAssembledEvent) once it holds the complete binlog split. The coordinator schedules the release in snapshotState() and performs it in notifyCheckpointComplete() after the covering checkpoint completes, so the binlog split assignment is always checkpoint-covered and can never be added back into an emptied assigner. Release is skipped when scan.newly-added-table is enabled, and the state/serializer format is unchanged (a released state is just empty maps). The assembled event has to be tied to a specific assignment. Flink does not drop source events from a failed attempt, and the event carried no identity, so after a reader failover a leftover event from the dead attempt could reach the coordinator and re-arm the release while the new reader was still pulling meta groups, which spans several checkpoints for a large table. The release would then fire and the new reader's next meta request would come back empty. Each assignment now has a generation that bumps on every add-back; the coordinator stamps it on the meta groups it serves and the reader echoes it back, so release only happens when the two match and a stale event is ignored. A split small enough to ship inline requests no meta groups, so its reader reports a sentinel the coordinator always accepts, since an inline reader never needs the metadata. The release state lives only in the singleton coordinator and there is a single binlog split, so parallel readers need no handling beyond this failover case. Tests cover the release protocol (assembled event plus a completed checkpoint, no release without the event or before the checkpoint completes, no release with newly-added-table on, add-back resetting the schedule, restore from a released light state), the stale-event case where an old-generation event after a failover must not trigger release, the inline-split case where release still happens after a generation bump, a reader-side check that the assembled event carries the served generation or the inline sentinel, a serializer round-trip of the released light state, and the assembled event being a no-op for a binlog-only assigner. Report URL: https://github.com/apache/flink-cdc/actions/runs/26727972391 With regards, GitHub Actions via GitBox
