[
https://issues.apache.org/jira/browse/KAFKA-9845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Egerton updated KAFKA-9845:
---------------------------------
Description:
The config provider mechanism doesn't work if used for the {{plugin.path}}
property of a standalone or distributed Connect worker. This is because the
{{Plugins}} instance which performs plugin path scanning is created using the
raw worker config, pre-transformation (see
[ConnectStandalone|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectStandalone.java#L79]
and
[ConnectDistributed|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectDistributed.java#L91]).
Unfortunately, because config providers are loaded as plugins, there's a
circular dependency issue here. The {{Plugins}} instance needs to be created
_before_ the {{DistributedConfig}}/{{StandaloneConfig}} is created in order for
the config providers to be loaded correctly, and the config providers need to
be loaded in order to perform their logic on any properties (including the
{{plugin.path}} property).
There is no clear fix for this issue in the code base, and the only known
workaround is to refrain from using config providers for the {{plugin.path}}
property.
A couple improvements could potentially be made to improve the UX when this
issue arises:
was:
The config provider mechanism doesn't work if used for the {{plugin.path}}
property of a standalone or distributed Connect worker. This is because the
{{Plugins}} instance which performs plugin path scanning is created using the
raw worker config, pre-transformation (see
[ConnectStandalone|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectStandalone.java#L79]
and
[ConnectDistributed|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectDistributed.java#L91]).
Unfortunately, because config providers are loaded as plugins, there's a
circular dependency issue here where the {{Plugins}} instance needs to be
created _before_ the {{DistributedConfig}}/{{StandaloneConfig}} is created in
order for the config provider logic to take place, and
> plugin.path property does not work with config provider
> -------------------------------------------------------
>
> Key: KAFKA-9845
> URL: https://issues.apache.org/jira/browse/KAFKA-9845
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.3.0, 2.4.0, 2.3.1, 2.5.0, 2.4.1
> Reporter: Chris Egerton
> Priority: Major
>
> The config provider mechanism doesn't work if used for the {{plugin.path}}
> property of a standalone or distributed Connect worker. This is because the
> {{Plugins}} instance which performs plugin path scanning is created using the
> raw worker config, pre-transformation (see
> [ConnectStandalone|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectStandalone.java#L79]
> and
> [ConnectDistributed|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectDistributed.java#L91]).
> Unfortunately, because config providers are loaded as plugins, there's a
> circular dependency issue here. The {{Plugins}} instance needs to be created
> _before_ the {{DistributedConfig}}/{{StandaloneConfig}} is created in order
> for the config providers to be loaded correctly, and the config providers
> need to be loaded in order to perform their logic on any properties
> (including the {{plugin.path}} property).
> There is no clear fix for this issue in the code base, and the only known
> workaround is to refrain from using config providers for the {{plugin.path}}
> property.
> A couple improvements could potentially be made to improve the UX when this
> issue arises:
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
