[
https://issues.apache.org/jira/browse/KAFKA-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-20145:
-----------------------------------
Description:
The new consumer (async, KIP-848) sends ack back to the broker whenever it
completes a reconciliation. But it also sends acks when receiving the same
target assignment (to cover the case where a member may get the same assignment
after being fenced)
To achieve this last bit, there is logic in the client state machine
reconciliation, to ack even if no change/progress. That would be ok when the
reconciliation was only triggered when a HB with a new assignment was received
from the GC, but with reconciliations triggered from the poll loop this
condition may generate noisy acks (harmless but unneeded).
At the moment, the consumer that gets a topicID-1 to reconcile and cannot
resolve metadata, will alternate between reconciling-ack-reconciling
continuously (acking back a same assignment, but triggered from poll, not
because the GC sends a new one)
Should be a minor fix and test
This is the logic that generates the ack on reconciliation attempts
[https://github.com/apache/kafka/blob/1515fddef07598ea56508fb82d2fb3546a52f831/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java#L845-L852]
(needed if the reconciliation attempt is triggered from a HB, but not needed
if triggered from poll I imagine)
was:
We need to send acks when receiving the same target assignment from GC, to
cover the case where a member may get the same assignment after being fenced
To achieve this, there is logic in the client state machine reconciliation, to
ack even if no change/progress. That would be ok when the reconciliation was
only triggered when a HB with a new assignment was received from the GC, but
with reconciliations triggered from the poll loop this condition may generate
noisy acks (harmless but unneeded).
At the moment, the consumer that gets a topicID-1 to reconcile and cannot
resolve metadata, will alternate between reconciling-ack-reconciling
continuously (acking back a same assignment, but triggered from poll, not
because the GC sends a new one)
Should be a minor fix and test
This is the logic that generates the ack on reconciliation attempts
[https://github.com/apache/kafka/blob/1515fddef07598ea56508fb82d2fb3546a52f831/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java#L845-L852]
(needed if the reconciliation attempt is triggered from a HB, but not needed
if triggered from poll I imagine)
> Improve consumer HB ack on unchanged target assignment
> ------------------------------------------------------
>
> Key: KAFKA-20145
> URL: https://issues.apache.org/jira/browse/KAFKA-20145
> Project: Kafka
> Issue Type: Task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Minor
>
> The new consumer (async, KIP-848) sends ack back to the broker whenever it
> completes a reconciliation. But it also sends acks when receiving the same
> target assignment (to cover the case where a member may get the same
> assignment after being fenced)
> To achieve this last bit, there is logic in the client state machine
> reconciliation, to ack even if no change/progress. That would be ok when the
> reconciliation was only triggered when a HB with a new assignment was
> received from the GC, but with reconciliations triggered from the poll loop
> this condition may generate noisy acks (harmless but unneeded).
> At the moment, the consumer that gets a topicID-1 to reconcile and cannot
> resolve metadata, will alternate between reconciling-ack-reconciling
> continuously (acking back a same assignment, but triggered from poll, not
> because the GC sends a new one)
> Should be a minor fix and test
> This is the logic that generates the ack on reconciliation attempts
> [https://github.com/apache/kafka/blob/1515fddef07598ea56508fb82d2fb3546a52f831/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java#L845-L852]
> (needed if the reconciliation attempt is triggered from a HB, but not needed
> if triggered from poll I imagine)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
