Hi we usually see close to wire-speed with rsync or other tools...
On 1/2/20 4:26 PM, Michael Di Domenico wrote: > i'm using 'rsync -rav --progress --stats -x --inplace > --compress-level=0 /dir1/ /dir2/' but each file (which is 100's of > GB's) is getting choked at 100MB/sec Hmm, Isn't this rsync running locally? What I would try: On one host (usually the receiver), start the rsync daemon with an appropriate small rsyncd.conf file and then from the other end try pushing into it, e.g. rsyncd.conf: [test] path = /path/to/lustre/storage uid = user gid = group list = yes hosts allow = * pid file = /run/rsync.pid read only = no and on the sender: rsync -avx --progress --stats /path/to/local/dir rsync://targetmachine/test/ What does this result in? Cheers Carsten -- Dr. Carsten Aulbert, Max Planck Institute for Gravitational Physics, Callinstraße 38, 30167 Hannover, Germany Phone: +49 511 762 17185
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
