ryannedolan commented on pull request #9589:
URL: https://github.com/apache/kafka/pull/9589#issuecomment-728354718
> The code seems to explicitly allow the connector to be created even when
the link is disabled.
> @ryannedolan maybe you could clarify?
My intention was to ensure that MirrorHeartbeatConnector always runs, even
when a link/flow is not otherwise needed. This is because
MirrorHeartbeatConnector is most useful when it emits to all clusters, not just
those clusters targeted by some flow.
For example, in a two-cluster environment with only A->B replicated, it is
nice to have heartbeats emitted to A s.t. they get replicated to B. Without a
Herder targeting A, there can be no MirrorHeartbeatConnector emitting
heartbeats there, and B will see no heartbeats from A.
I know that some vendors/providers use heartbeats in this way, e.g. for
discovering which flows are active and healthy. And I know that some
vendors/providers don't use heartbeats at all, or use something else to send
them (instead of MirrorHeartbeatConnector). Hard to say whether anything would
break if we nixed these extra herders without addressing the heartbeats that
would go missing.
IMO, we'd ideally skip creating the A->B herder whenever
A->B.emit.heartbeats.enabled=false (defaults to true) and A->B.enabled=false
(defaults to false). A top-level emit.heartbeats.enabled=false would then
disable heartbeats altogether, which would trivially eliminate the extra
herders. N.B. this would just be an optimization and wouldn't required a KIP,
IMO.
Ryanne
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
