[
https://issues.apache.org/jira/browse/KAFKA-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860507#comment-17860507
]
M Mehrtens commented on KAFKA-16364:
------------------------------------
This is especially relevant for cluster migration scenarios when consumer group
offsets should be replicated with a minimum latency. Ideally, consumer groups
could detach from the source cluster and attach to the target cluster without
needing to process a batch of duplicate messages. This would require offset
sync/translation for inactive consumer groups - which seems like a pretty
reasonable addition, especially if the consumer groups were previously synced.
> MM2 High-Resolution Offset Translation
> --------------------------------------
>
> Key: KAFKA-16364
> URL: https://issues.apache.org/jira/browse/KAFKA-16364
> Project: Kafka
> Issue Type: New Feature
> Components: mirrormaker
> Reporter: Greg Harris
> Priority: Minor
> Labels: needs-kip
>
> The current OffsetSyncStore implementation
> [https://github.com/apache/kafka/blob/8b72a2c72f09838fdd2e7416c98d30fe876b4078/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncStore.java#L57]
> stores a sparse index of offset syncs. This attempts to strike a balanced
> default behavior between offset translation availability, memory usage, and
> throughput on the offset syncs topic.
> However, this balanced default behavior is not good enough in all
> circumstances. When precise offset translation is needed away from the end of
> the topic, such as for consumer groups with persistent lag, offset
> translation can be more precise. Users should have a way to configure
> high-precision offset translation, either through additional memory usage or
> re-reading the offset syncs topic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
