The whitepaper was interesting. Single core VMs might be your best bet. On Thu, Jan 2, 2020, 8:48 AM Michael Di Domenico <mdidomeni...@gmail.com> wrote:
> i'll check it, but keep in mind. i'm not copying files between two > servers, but rather between two directories on the same server. > > ideally if rsync is still using ssh under the covers in my scenario, > i'm hopeful hpn-ssh might alleviate the bottleneck condition. if it's > not i'm back to square one. > > On Thu, Jan 2, 2020 at 11:42 AM Alex Chekholko <a...@calicolabs.com> > wrote: > > > > Hi Michael, > > > > I would recommend trying 'bbcp' before 'hpn-ssh' as the latter will > really only benefit you for high-latency links, e.g. across country. > > > > Put the bbcp binary on both sides and try it out. If you don't have a > way to install bbcp into a system $PATH, you can specify the absolute path > to the binary. Random link with examples here: > > https://www.nics.tennessee.edu/computing-resources/data-transfer/bbcp > > > > Regards, > > Alex > > > > On Thu, Jan 2, 2020 at 8:32 AM Michael Di Domenico < > mdidomeni...@gmail.com> wrote: > >> > >> just to further the discussion and for everyone's education i found > >> this whitepaper, which seems to confirm what i see > >> > >> > https://www.intel.com/content/dam/support/us/en/documents/network/sb/fedexcasestudyfinal.pdf > >> > >> maybe hpn-ssh is something i can work into my process > >> > >> > >> On Thu, Jan 2, 2020 at 10:26 AM Michael Di Domenico > >> <mdidomeni...@gmail.com> wrote: > >> > > >> > does anyone know or has anyone gotten rsync to push wire speed > >> > transfers of big files over 10G links? i'm trying to sync a directory > >> > with several large files. the data is coming from local disk to a > >> > lustre filesystem. i'm not using ssh in this case. i have 10G > >> > ethernet between both machines. both end points have more then > >> > enough spindles to handle 900MB/sec. > >> > > >> > i'm using 'rsync -rav --progress --stats -x --inplace > >> > --compress-level=0 /dir1/ /dir2/' but each file (which is 100's of > >> > GB's) is getting choked at 100MB/sec > >> > > >> > running iperf and dd between the client and the lustre hits 900MB/sec, > >> > so i fully believe this is an rsync limitation. > >> > > >> > googling around hasn't lent any solid advice, most of the articles are > >> > people that don't check the network first... > >> > > >> > with the prevalence of 10G these days, i'm surprised this hasn't come > >> > up before, or my google-fu really stinks. which doesn't bode well > >> > given its the first work day of 2020 :( > >> _______________________________________________ > >> Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin > Computing > >> To change your subscription (digest mode or unsubscribe) visit > https://beowulf.org/cgi-bin/mailman/listinfo/beowulf > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > https://beowulf.org/cgi-bin/mailman/listinfo/beowulf >
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
