On Mon, Aug 13, 2018 at 2:07 AM Patrick Reedy <[email protected]> wrote:
> I’m struggling with a pipeline stage that needs to complete quickly, in less > than a second, but sometimes is taking 5 or 6 seconds. > > In this pipeline stage each of the 50 nodes have to send their to piece of > data to all of the 100 main workers - so 50 nodes each sending 100 pieces of > data with each piece about 1.5M bytes. The main workers need to collect all > of the pieces of data before the pipeline can advance to the next stage. What kind of network interfaces do you have? When this stage runs how much data total does each physical interface send/retrieve? Could you be saturating the network? -- - Justin _______________________________________________ zeromq-dev mailing list [email protected] https://lists.zeromq.org/mailman/listinfo/zeromq-dev
