[
https://issues.apache.org/jira/browse/KAFKA-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangning.li updated KAFKA-19735:
--------------------------------
Description:
Suppose that no new messages have been written to the target topic for a long
time, for example, the producer has not generated new data for nearly an hour.
On the consumer side, when the user sets enable.auto.commit=true, the
OFFSET_COMMIT request will still be repeatedly submitted every 5 seconds. This
not only wastes network resources but also increases the burden on
__consumer_offset.
Therefore, maybe we can add a cache for committed offsets to check if the
currently committed offset is consistent with the most recently successfully
committed offset. If they are the same, this offset commit can be ignored. Of
course, all the above applies to the subscribe mode.
Advantages: * Reduce unnecessary network requests
* Alleviate the processing pressure on the broker side
* Relieve the pressure of log cleaning for __consumer_offset
was:
Suppose that no new messages have been written to the target topic for a long
time, for example, the producer has not generated new data for nearly an hour.
On the consumer side, when the user sets enable.auto.commit=true, the
OFFSET_COMMIT request will still be repeatedly submitted every 5 seconds. This
not only wastes network resources but also increases the burden on
__consumer_offset.
Therefore, maybe we can add a cache for committed offsets to check if the
currently committed offset is consistent with the most recently successfully
committed offset. If they are the same, this offset commit can be ignored. Of
course, all the above applies to the subscribe mode.
> Add automatic commit offset caching in subscribe mode
> -----------------------------------------------------
>
> Key: KAFKA-19735
> URL: https://issues.apache.org/jira/browse/KAFKA-19735
> Project: Kafka
> Issue Type: Improvement
> Reporter: kangning.li
> Assignee: kangning.li
> Priority: Minor
>
> Suppose that no new messages have been written to the target topic for a long
> time, for example, the producer has not generated new data for nearly an
> hour. On the consumer side, when the user sets enable.auto.commit=true, the
> OFFSET_COMMIT request will still be repeatedly submitted every 5 seconds.
> This not only wastes network resources but also increases the burden on
> __consumer_offset.
>
> Therefore, maybe we can add a cache for committed offsets to check if the
> currently committed offset is consistent with the most recently successfully
> committed offset. If they are the same, this offset commit can be ignored. Of
> course, all the above applies to the subscribe mode.
>
> Advantages: * Reduce unnecessary network requests
> * Alleviate the processing pressure on the broker side
> * Relieve the pressure of log cleaning for __consumer_offset
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
