[
https://issues.apache.org/jira/browse/KAFKA-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148818#comment-17148818
]
Boyang Chen commented on KAFKA-10219:
-------------------------------------
Thanks for the proposal, the use case you proposed is reasonable. However, we
need to better clarify the feature we are going to introduce and the challenges
we are facing, such as:
1. What does "multiple clusters" suggest? Do we support all input topics in
cluster A and all output topics in cluster B, or a mixing of topics in random
cluster A, B, C which needs to be automatically detected by Streams?
2. How do we allocate internal topics? Which cluster should the
changelog/repartition topics go to, input topic cluster, or the output one?
3. How do we support Exactly-once? Right now the entire framework assumes a
single cluster context. When switching to multiple cluster, we could no longer
guarantee exactly-once because we may spam our transaction across multiple
clusters, and we don't have a centralized coordinator to track the progress.
> KStream API support for multiple cluster broker
> -----------------------------------------------
>
> Key: KAFKA-10219
> URL: https://issues.apache.org/jira/browse/KAFKA-10219
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Sachin Kurle
> Priority: Major
>
> we are trying to consume from cluster A broker from KStream api and produce
> to cluster B broker.. we have configuration as boot strap server in consumer
> and producer configuration but kstream api is picking randomly bootstrap
> server cluster A or B
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
