On Fri, 2008-08-08 at 11:59 -0400, Joe Landman wrote: > > The transmission of a single package is fast. The time of passing a > > particular > > package through the whole chain of nodes is short compared with time of the > > entire copying process. E.g., using jumbo frames a package can have the > > size of ca 10kB. > > In Gb network the transmission time of a single package between nodes is > > of the order of 0.1 ms. Even in a cluster with 1024 nodes it takes > > in an ideal case just 0.1s to pass a package from node n1 through all nodes > > to n1024. > > > > On each node the package is stored and, in the end, one reassembles the > > file. > > For big files (size >> 10Mb) the required time is approximately > > the same as one needs for copying the file between two nodes plus 0.1s. > > > > One needs basically a daemon which handles copying requests and establishes > > the connection to next node in the chain. > > > > Has somebody written such a tool? > > I saw something like this several years ago.
I've written a couple of them over the years, one as a way of copying a new OS between nodes and I started one more recently as a general purpose copy tool. I've always thought that something like this should be written in MPI and simply make use of MPI_Bcast() and MPI_Addreduce() to do the hard part, it's very easy to write a version that copies a single file, slightly harder to do multiple files but still not rocket science. Basically efficient broadcast isn't as easy to make generic as it seems, why waste time even trying when you can get MPI to do all tricky bits like work out toplogy/starting deamons/security and in all probability do it better to boot? Ashley Pittman. _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
