lucasbru opened a new pull request, #15511:
URL: https://github.com/apache/kafka/pull/15511
The goal of this PR is to change the following internals of the
reconciliation:
- Introduce a "local epoch" to the local target assignment. When a new
target is received by the server, we compare it with the current value. If it
is the same, no change. Otherwise, we bump the local epoch and store the new
target assignment. Then, on the reconciliation, we also store the epoch in the
reconciled assignment and keep using target != current to trigger the
reconciliation.
- When we are not in a group (we have not received an assignment), we use
null to represent the local target assignment instead of an empty list, to
avoid confusions with an empty assignment received by the server. Similarly, we
use null to represent the current assignment, when we haven't reconciled the
assignment yet.
- We also carry the new epoch into the request builder to ensure that we
report the owned partitions for the last local epoch.
- To address
[KAFKA-16312](https://issues.apache.org/jira/browse/KAFKA-16312) (call
onPartitionsAssigned on empty assignments after joining), we apply the initial
assignment returned by the group coordinator (whether empty or not) as a normal
reconciliation. This avoids introducing another code path to trigger rebalance
listeners - reconciliation is the only way to transition to STABLE. The
unneeded parts of reconciliation (autocommit, revocation) will be skipped in
the existing. Since a lot of unit tests assumed that not reconciliation
behavior is invoked when joining the group with an empty assignment, this
required a lot of the changes in the unit tests.
## Testing
These changes allow the new consumer to pass the first 10 system tests. We
piggy-back a minor change to the `HeartbeatManager` that are required for those
system tests as well: always send `rebalanceTimoutMs`, `subscriptions` when
(re-)joining.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
