Stock RH 7 version, rsync-3.1.2-6.el7_6.1.x86_64. We've tried a number
of recompiles. gcc, Intel. The only thing between identical compiles
was the md4 vs md5.
/bin/rsync -lptgoDAH -v --numeric-ids -d --relative --delete
--delete-after --files-from=...
I'm not asking for help. Just if anyone had attempted to change the
algorithm into something much faster.
I refer you to this project https://cyan4973.github.io/xxHash/ where
there is a table of speeds. Regardless of what anyone might speculate,
we are pursuing this route of changing out the algorithm. Maybe it's
all for naught. Maybe it isn't. But in a few weeks hopefully we'll
have determined.
Thanks all,
Bill
On 6/18/19 9:02 AM, Ellis H. Wilson III wrote:
On 6/18/19 6:59 AM, Bill Wichser wrote:
Just for clarity here, we are NOT using the -c option. The checksums
happen whenever there is a transfer between the rsync source and the
rsyncd on the other end.
...snip...
This is not some trivial rsync running at the top level. There is
code we wrote as well as integration with Jenkins. When we recompiled
rsync using MD4 instead of the MD5 we see a 20% increase in
performance across the board. This is what sparked my question.
We need more details to be of much use:
1. Specific rsync version and command line used.
2. Compilation options both normally and with your md4 changes.
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
