[
https://issues.apache.org/jira/browse/KAFKA-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans reopened KAFKA-20332:
------------------------------------
Reopened for minor gap with wakeup. Small PR in review
> Ensure app thread not collecting records for partitions being revoked
> ---------------------------------------------------------------------
>
> Key: KAFKA-20332
> URL: https://issues.apache.org/jira/browse/KAFKA-20332
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 4.2.0
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Blocker
> Fix For: 4.3.0, 4.2.1
>
>
> With the changes to not fully wait on a PollEvent on the asyncConsumer app
> thread (4.2), there could be a race if records are buffered for a partition
> and the partition gets revoked, with the app thread potentially collecting
> records from the buffer while the partition is being revoked (ending with
> records being returned to the app after commit/revocation)
> This sequence: hb received revoking partitions, reconciliation identified but
> not triggered in background needing commit, app thread sends AsyncPoll that
> triggers the reconciliation with commit, but moves on to collect records from
> the buffer (fetchable if the pending reconciliation to revoke hasn't made it
> to the mark to stop fetching yet). So this could lead to records loaded into
> memory on the app thread, background then completing the revocation, app
> thread returning records for a revoked partition.
> I expect this only affects 4.2 due to the change to not fully wait on the
> poll event (we used to wait before, so would never make it to collect when
> there was a reconciliation attempt, ensuring we hit the mark to stop fetching
> before proceeding in the app thread).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
