Greg Harris created KAFKA-16364:
-----------------------------------
Summary: MM2 High-Resolution Offset Translation
Key: KAFKA-16364
URL: https://issues.apache.org/jira/browse/KAFKA-16364
Project: Kafka
Issue Type: New Feature
Components: mirrormaker
Reporter: Greg Harris
The current OffsetSyncStore implementation
[https://github.com/apache/kafka/blob/8b72a2c72f09838fdd2e7416c98d30fe876b4078/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncStore.java#L57]
stores a sparse index of offset syncs. This attempts to strike a balanced
default behavior between offset translation availability, memory usage, and
throughput on the offset syncs topic.
However, this balanced default behavior is not good enough in all
circumstances. When precise offset translation is needed away from the end of
the topic, such as for consumer groups with persistent lag, offset translation
can be more precise. Users should have a way to configure high-precision offset
translation, either through additional memory usage or re-reading the offset
syncs topic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
