[
https://issues.apache.org/jira/browse/KAFKA-15203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849814#comment-17849814
]
Ganesh Sadanala edited comment on KAFKA-15203 at 5/28/24 1:17 AM:
------------------------------------------------------------------
[~chia7712] There are 2 ways to invoke the Kafka Connect Plugin discovery.
1. The first is the core application flow(Plugins.initLoaders(...) which is
used by Worker.startConnector(...)). I see that initLoaders is still using
`ReflectionScanner` along with `ServiceLoader` to load plugins. The use case is
to completely transition this to `ServiceLoader`. [~gharris1727] Is
`ReflectionScanner` still needed here?
{code:java}
// public PluginScanResult initLoaders(Set<PluginSource> pluginSources,
PluginDiscoveryMode discoveryMode) {
PluginScanResult empty = new PluginScanResult(Collections.emptyList());
PluginScanResult serviceLoadingScanResult;
try {
serviceLoadingScanResult = discoveryMode.serviceLoad() ?
new ServiceLoaderScanner().discoverPlugins(pluginSources) :
empty;
} catch (Throwable t) {
throw new ConnectException(String.format(
"Unable to perform ServiceLoader scanning as requested by
%s=%s. It may be possible to fix this issue by reconfiguring %s=%s",
WorkerConfig.PLUGIN_DISCOVERY_CONFIG, discoveryMode,
WorkerConfig.PLUGIN_DISCOVERY_CONFIG,
PluginDiscoveryMode.ONLY_SCAN), t);
}
PluginScanResult reflectiveScanResult = discoveryMode.reflectivelyScan() ?
new ReflectionScanner().discoverPlugins(pluginSources) : empty;
PluginScanResult scanResult = new
PluginScanResult(Arrays.asList(reflectiveScanResult, serviceLoadingScanResult));
maybeReportHybridDiscoveryIssue(discoveryMode, serviceLoadingScanResult,
scanResult);
delegatingLoader.installDiscoveredPlugins(scanResult);
return scanResult;
} {code}
2. The terminal invoking tool/script `connect-plugin-path.sh` which uses the
`org.apache.kafka.tools.ConnectPluginPath` to make use of both the Reflections
and ServiceLoader mechanism to load the plugins. Since this a manual invoke
(which I assume), there is no harm in using the Reflections for loading other
plugins which are not required for the connect and/or not having the manifest
files.
I think this is the reason why [~gharris1727] said about it in the first
comment: "The long term plan will be to make the reflections library less
necessary via KAFKA-14627, and then the usage of reflections can be limited to
the connect-plugin-path.sh script. We have not yet decided on a deprecation
schedule and corresponding sunset date on the usage of this library."
I am still going through the codebase to see if any modifications are needed.
[~chia7712] To conclude, there is no need to remove the actual
ReflectionScanner class but I need to remove the ReflectionScanner from the
actual application flow above to not affect the script execution. If script
execution is not needed and can be removed, ReflectionScanner can be totally
removed.
These are my conclusions after researching the KPIs and codebase. I am waiting
for your replies so that I can remove unnecessary code part above and raise a
PR.
was (Author: JIRAUSER305566):
[~chia7712] There are 2 ways to invoke the Kafka Connect Plugin discovery.
1. The first is the core application flow(Plugins.initLoaders(...) which is
used by Worker.startConnector(...)). I see that initLoaders is still using
`ReflectionScanner` along with `ServiceLoader` to load plugins. The use case is
to completely transition this to `ServiceLoader`. [~gharris1727] Is
`ReflectionScanner` still needed here?
2. The terminal invoking tool/script `connect-plugin-path.sh` which uses the
`org.apache.kafka.tools.ConnectPluginPath` to make use of both the Reflections
and ServiceLoader mechanism to load the plugins. Since this a manual invoke
(which I assume), there is no harm in using the Reflections for loading other
plugins which are not required for the connect and/or not having the manifest
files.
I think this is the reason why [~gharris1727] said about it in the first
comment: "The long term plan will be to make the reflections library less
necessary via KAFKA-14627, and then the usage of reflections can be limited to
the connect-plugin-path.sh script. We have not yet decided on a deprecation
schedule and corresponding sunset date on the usage of this library."
I am still going through the codebase to see if any modifications are needed.
> Remove dependency on Reflections
> ---------------------------------
>
> Key: KAFKA-15203
> URL: https://issues.apache.org/jira/browse/KAFKA-15203
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Reporter: Divij Vaidya
> Assignee: Ganesh Sadanala
> Priority: Major
> Labels: newbie
> Fix For: 5.0.0
>
>
> We currently depend on reflections library which is EOL. Quoting from the
> GitHub site:
> _> Please note: Reflections library is currently NOT under active development
> or maintenance_
>
> This poses a supply chain risk for our project where the security fixes and
> other major bugs in underlying dependency may not be addressed timely.
> Hence, we should plan to remove this dependency.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
