kumarpritam863 opened a new pull request, #12727: URL: https://github.com/apache/iceberg/pull/12727
**Issue** In rare scenarios, two Kafka Connect jobs may unintentionally be configured with the same consumer group ID. When this happens, one of the jobs may silently hang — even though the consumer group appears stable — because it cannot find any member consuming from its assigned source topics (i.e., the member with the lexicographically smallest partition) to elect itself as coordinator. **Root Cause** 1. The connector logic assumes each job is isolated by group ID. 2. If two jobs share a group ID but operate on disjoint sets of topics, only one job gets assigned partitions, while the other finds none and cannot proceed. 3. Because no error is thrown and the group is technically "stable", the idle job just waits indefinitely. **Solution** This PR introduces validation and safeguards to detect and prevent this silent failure: 1. Topic Ownership Filtering Before performing coordinator election, the connector filters group members based on the topics this job is configured for (topics or topics.regex). Only members consuming from those topics are considered valid. 2. Client ID Collision Detection To catch potential overlaps between jobs sharing the same group ID, the PR also checks for duplicate consumer.override.client.id values within the filtered members. If a collision is detected, the job fails early with a clear error message. **Behavior After This Change** If two jobs accidentally share a consumer group but have overlapping client IDs, one will fail fast with an informative exception. **If two jobs share a group but are assigned mutually exclusive topics, the election logic will only consider relevant members — preventing silent hangs.** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
