Re: Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Ralph Goers
The ThreadPool for ConfigurationScheduler is tied to the Configuration. So it isn’t created until after plugins are loaded (its name should make that obvious). Also, the number of threads it configures the thread pool for is determined based on what it finds in the configuration. It is quite pos

Re: Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Ralph Goers
Notice that class path scanning only occurs when the following evaluates to true: if (builtInPlugins.isEmpty()) { We encourage the value of this to always be false. I think it would be good to log an info message indicating that package scanning is taking place and the application should be cor

Re: Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Volkan Yazıcı
I am not sure if spawning up and tearing down a new thread pool for each call would be a good idea. On Mon, Mar 8, 2021 at 10:46 PM Gary Gregory wrote: > We do have a thread pool factory that decorates thread names with > "log4j" IIRC... > > On Mon, Mar 8, 2021 at 4:29 PM Volkan Yazıcı > wrote:

Re: Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Gary Gregory
We do have a thread pool factory that decorates thread names with "log4j" IIRC... On Mon, Mar 8, 2021 at 4:29 PM Volkan Yazıcı wrote: > > Do we have a Log4j-specific ExecutorService somewhere we can hook into > PluginManager? > > On Mon, Mar 8, 2021 at 10:24 PM Carter Kozak wrote: > > > Please d

Re: Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Volkan Yazıcı
Do we have a Log4j-specific ExecutorService somewhere we can hook into PluginManager? On Mon, Mar 8, 2021 at 10:24 PM Carter Kozak wrote: > Please don't use stream.parallel! It executes tasks on the ForkJoin pool > which implements work-stealing, and can result in expensive long-running > operat

Re: Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Carter Kozak
Please don't use stream.parallel! It executes tasks on the ForkJoin pool which implements work-stealing, and can result in expensive long-running operations from other components in the system that we don't expect blocking our call (anything using the fork-join pool, all stream.parallel callers)

Parallelization of PluginManager#collectPlugins()

2021-03-08 Thread Volkan Yazıcı
PluginManager#collectPlugins() performs quite some package scanning sequentially. I have the impression that this operation can simply be parallelized as follows: Stream .of(inputs) .flatMap(input -> Stream .of(ops) .map(op -> new Object[]{op, input}))