The GitHub Actions job "Flink Connector Kafka CI" on 
flink-connector-kafka.git/fix-idle-reader-no-more-splits has failed.
Run started by GitHub user slachiewicz (triggered by slachiewicz).

Head commit for run:
0fe34760b94e44bed8934b3f8d13fbfad40695c1 / Sylwester Lachiewicz 
<[email protected]>
[FLINK-40362][connector/kafka] Fix idle reader hang when no-more-splits 
precedes metadata update

An idle reader (subtask with no assigned splits) that receives the 
no-more-splits
signal before the reply to GetMetadataUpdateEvent never forwards it to the 
sub-readers
created by that update: the re-notification in handleSourceEvents was guarded by
!pendingSplits.isEmpty(), which is false in exactly this case. The reader then
returns NOTHING_AVAILABLE indefinitely and a bounded job never finishes.

Re-deliver the no-more-splits signal after processing the reader's first 
metadata update
(captured via !isActivelyConsumingSplits before the flag flips). Replaying only 
on the
first metadata update ensures that an active reader waiting for splits on newly 
added
clusters/topics does not terminate prematurely before those splits and the 
subsequent
no-more-splits signal arrive. notifyNoMoreSplits() is idempotent and only fires 
when
isNoMoreSplits is true.

Verified:
- 
DynamicKafkaSourceReaderTest#testIdleReaderFinishesWhenNoMoreSplitsArrivesBeforeMetadata:
  fails on main (NOTHING_AVAILABLE), passes with this change (END_OF_INPUT).
- 
DynamicKafkaSourceReaderTest#testActiveReaderWaitsForNewSplitsAfterMetadataChange:
  passes on main, fails if re-delivery is unconditional on every metadata 
update,
  passes with the firstMetadataUpdate guard.

Generated-by: Claude Fable 5

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/34370338998

With regards,
GitHub Actions via GitBox

Reply via email to