lucasbru opened a new pull request, #20600: URL: https://github.com/apache/kafka/pull/20600
Streams groups sometimes describe as NOT_READY when STABLE. That is, the group is configured and all topics exist, but when you use LIST_GROUP and STREAMS_GROUP_DESCRIBE, the group will show up as not ready. The root cause seems to be that https://github.com/apache/kafka/pull/19802 moved the creation of the soft state configured topology from the replay path to the heartbeat. This way, LIST_GROUP and STREAMS_GROUP_DESCRIBE, may not show the configured topology, because the configured topology that is created in the heartbeat is "thrown away", and the new group is recreated on the replay-path. To reflect a consistent view of the topology via LIST_GROUP and STREAMS_GROUP_DESCRIBE, we need to store additional information in the consumer offset topic. In particular, we need to store at least whether a topology was validated against the current topic metadata, as this defines whether a group is in STABLE and not in NOT_READY. This change adds a new field `validatedTopologyEpoch` to the metadata of the group, which stores precisely this information. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
