On 10/3/2013 9:52 AM, Dominic Raferd wrote:
There have been discussions about verification speeds and issues here
before. I think you are right that the CPU core is a bottleneck as
rdiff-backup only uses one core when running a verification.
Verifications do use temporary space and can use a lot of it even though
the temporary files never seem to be visible in the filesystem. I would
advise an explicit --tempdir setting, and a different spindle with lots
of space (and speed) would be ideal.
Yeah, I think the verifications definitely can benefit from having a
spindle dedicated to /tmp (or --tempdir setting). We added a 2nd set of
small fast spindles, moved /tmp over, and it seems to have dramatically
sped up the verify speed. With "atop" we can see rates of 80-120 MB/s
being written to the /tmp file system now that it is on its own spindle.
So it's getting a lot of use.
From what I can tell, we're still limited by the spindle speeds of both
the place where the rdiff-backup directory is stored, and also the speed
of the /tmp (or --tempdir) location.
Our Opteron 6348 (12-core 2.8GHz) doesn't seem to have trouble
calculating the SHA1 hashes that rdiff-backup uses.
I don't have hard numbers on rdiff-backup itself, but our script to
rsync the rdiff-backup files off to an external USB3 drive now takes
about 12 hours instead of 18 hours. It also does --verify of the source
directory prior to the rsync, then does a --verify of the target
directory after the rsync. I have to examine things again to see if I
can cut that time window down to about 4-5 hours.
The next step for us will be to move the rdiff-backup directories to a
PV with more and faster spindles.
_______________________________________________
rdiff-backup-users mailing list at [email protected]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
