This has bothered me for a long time, so I decided to ask possibilities.
I know NiFi has decided that multiple named input ports are not worth
effort.
Still sometimes they are very useful.
<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/file/n1995/Named_inputs_2015-06-21.png>
 
Please see:
http://imgur.com/eGvuwR0

Instead of creating some data structure which holds multiple parts,
BuildResponse gets different parts from different parts from different
inputs.
This is much better.
Also for many business software developers which have used to use procedural
approach named input ports are more understandable. Also this quickly show
data flow by data parts name and ports clearly show which data they consume.

How difficult is to create experimental feature which supports multiple
named input ports?
What effort this requires?

My thoughts:
1. Initially hopefully NiFi UI does need to show multiple named ports.
I don't know much about NiFi UI, so I may not able to implement such
functionality at reasonable time.

2. May be flow XML template can be used to hold multiple named ports
information.
Does /nifi-framework-core/src/main/resources/FlowConfiguration.xsd define
flow XML structure? Does this allow easily add multiple named input ports
information?

3. Custom version of StandardProcessScheduler is needed. Custom scheduler
will wait until all input ports contain required data (using correlation
Id).
And only after what will invoke processor onTrigger method.
Or even better some custom processor method which has exactly as many input
parameters as processor have input ports.

Hopefully experimental changes won't break some NiFi functionality, lineage,
repositories, etc.

Is this totally crazy?
I am ready to implement some experimental functionality, but don't know what
is the best way to start.
This does not need to be part of official NiFi distribution.
But can be when community consider this useful.

Thanks
Toivo




--
View this message in context: 
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/How-to-implement-Scatter-Gather-tp1944p1995.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at 
Nabble.com.

Reply via email to