Lianet Magrans created KAFKA-17154:
--------------------------------------
Summary: New consumer subscribe may join group without a call to
consumer.poll
Key: KAFKA-17154
URL: https://issues.apache.org/jira/browse/KAFKA-17154
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 3.8.0
Reporter: Lianet Magrans
Fix For: 4.0.0
With the new consumer, a call to consumer.subscribe will update the client
subscription state via a SubscriptionChangedEvent. This will be picked up by
the background thread, and will send a Heartbeat RPC on the next poll of the
background thread (without requiring a consumer.poll). This is a change in
behaviour with the legacy consumer, that breaks the consumer#subscribe contract
for "rebalances will only occur during an active call to \{@link
#poll(Duration)}", so it should be fixed.
In the legacy its a similar principle but different behaviour: subscribe
changes the subscription state, and the coordinator picks it up to send the
JoinRequest on the next poll of the coordinator (but this happens only as part
of a consumer.poll)
We should make the new consumer join (send HB to join) only on consumer.poll
after a call to subscribed. We could consider having the
SubscriptionChangedEvent only signal to the background that the subscription
changed without doing the transition to JOINING that triggers the HB
(membershipMgr#onSubscriptionUpdated could simply flip a flag, no transition),
and then do the actual transition to JOINING when processing a PollEvent if the
flag is on (flipping it back). (Just first simple approach that comes to mind,
we should think about it a bit more and consider other interactions that could
happen in between, like unsubscribe, close, etc).
We should add test coverage to ensure the common behaviour with the legacy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
