[
https://issues.apache.org/jira/browse/KAFKA-16514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sal Sorrentino updated KAFKA-16514:
-----------------------------------
Description:
Working with Kafka Streams 3.7.0, but may affect earlier versions as well.
When attempting to shutdown a streams application and leave the associated
consumer group, the supplied `leaveGroup` option seems to have no effect.
Sample code:
{code:java}
CloseOptions options = new CloseOptions().leaveGroup(true);
stream.close(options);{code}
The expected behavior here is that the group member would shutdown and leave
the group, immediately triggering a consumer group rebalance. In practice, the
rebalance happens after the appropriate timeout configuration has expired.
I understand the default behavior in that there is an assumption that any
associated StateStores would be persisted to disk and that in the case of a
rolling restart/deployment, the rebalance delay may be preferable. However, in
our application we are using in-memory state stores and standby replicas. There
is no benefit in delaying the rebalance in this setup and we are in need of a
way to force a member to leave the group when shutting down.
The workaround we found is to set an undocumented internal StreamConfig to
enforce this behavior:
{code:java}
props.put("internal.leave.group.on.close", true);
{code}
To state the obvious, this is less than ideal.
was:
Working with Kafka Streams 3.7.0, but may affect earlier versions as well.
When attempting to shutdown a streams application and leave the associated
consumer group, the supplied `leaveGroup` option seems to have no effect.
Sample code:
{code:java}
CloseOptions options = new CloseOptions().leaveGroup(true);
stream.close(options);{code}
The expected behavior here is that the group member would shutdown and leave
the group, immediately triggering a consumer group rebalance. In practice, the
rebalance happens after the appropriate timeout configuration has expired.
I understand the default behavior in that there is an assumption that ant
associated StateStores would be persisted to disk and that in the case of a
rolling restart/deployment, the rebalance delay may be preferable. However, in
our application we are using in-memory state stores and standby replicas. There
is no benefit in delaying the rebalance in this setup and we are in need of a
way to force a member to leave the group when shutting down.
The workaround we found is to set an undocumented internal StreamConfig to
enforce this behavior:
{code:java}
props.put("internal.leave.group.on.close", true);
{code}
To state the obvious, this is less than ideal.
> Kafka Streams: stream.close(CloseOptions) does not respect options.leaveGroup
> flag.
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-16514
> URL: https://issues.apache.org/jira/browse/KAFKA-16514
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Sal Sorrentino
> Priority: Major
>
> Working with Kafka Streams 3.7.0, but may affect earlier versions as well.
>
> When attempting to shutdown a streams application and leave the associated
> consumer group, the supplied `leaveGroup` option seems to have no effect.
> Sample code:
> {code:java}
> CloseOptions options = new CloseOptions().leaveGroup(true);
> stream.close(options);{code}
> The expected behavior here is that the group member would shutdown and leave
> the group, immediately triggering a consumer group rebalance. In practice,
> the rebalance happens after the appropriate timeout configuration has expired.
> I understand the default behavior in that there is an assumption that any
> associated StateStores would be persisted to disk and that in the case of a
> rolling restart/deployment, the rebalance delay may be preferable. However,
> in our application we are using in-memory state stores and standby replicas.
> There is no benefit in delaying the rebalance in this setup and we are in
> need of a way to force a member to leave the group when shutting down.
> The workaround we found is to set an undocumented internal StreamConfig to
> enforce this behavior:
> {code:java}
> props.put("internal.leave.group.on.close", true);
> {code}
> To state the obvious, this is less than ideal.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
