Please see http://www.enterpriseintegrationpatterns.com/BroadcastAggregate.html
My use case is little bit different but overall pattern is same. Instead of sending requests to different vendors some data must be calculated. Processors Calc_A, Calc_B, Calc_C and Calc_D calculate values A, B, C and D. All processors are independent from each other. Each processor use only data from request as input. Every request contains unique requestId which will be attached to every FlowFile during processing. How to implement this in NiFi? 1. Sequential processing Listener -> Calc_A -> Calc_B -> Calc_C -> Calc_D -> BuildResponse -> SendResponse Each Calc_* processor will add calculated value to FlowFile. So After Calc_B FlowFile contains A and B values. And After Calc_C FlowFile contains A, B and C values. BuildResponse incoming FlowFile contains A, B, C and D values and building response is easy. Sequential processing works but when Calc_* processors are slow overall service latency is too big. 2. Parallel processing After Listener we can send request FlowFile to all Calc_* processors simultaneously. Problem is how to aggregate values A, B, C and D from different processors? This question may belong to user list, but I thought solutions are probably not trivial and maybe some overall development is reasonable? Thanks Toivo -- View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/How-to-implement-Scatter-Gather-tp1944.html Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
