Hi,
I'm trying to use rdiff-backup to backup a bunch of servers. One
particular server contains about 160GB of data, but when I try to perform
the rdiff-backup it's saving the data at a measly 1MB/s.
Here's my configuration:
[server] <--ssh--> [backup-server]{encfs} <--nfs--> [freenas]
I ran a bunch of tests to try to figure out my bottlenecks.
I ran a bunch of dd tests (using dd if=/dev/zero bs=1M count=1000) on the
backup server. Going directly to FreeNAS via NFS (bybassing encfs) I get
50.2MB/s. If I run dd directly on the backup server (through encfs) I get
20.1MB/s. If I go over SSH from the backup server to the target server
and run the dd on the target server, then write to FreeNas through encfs
declines to 7.6MB/s.
Note that in those SSH tests, however, I forgot to turn off compression.
When I do that, the throughput for the dd test reduced to 6.6BM/s. (Note
that this is running simultaneously with a running rdiff-backup, so it's
possible that they are reducing performance).
Then I ran an scp test to the same target server; copying about 1.4GB of
photos. Files ranged in size from 10KB to 5MB. When run in standard mode
(displaying each file status) I got 4.4MB/s. Running in quiet mode I get
5.1MB/s.
So clearly the bottleneck is in rdiff-backup -- performance (IMHO) should
not be significantly slower than the last dd-over-ssh test. It appears
rdiff-backup is slowing me down by a factor of 5x throughput versus scp.
I found a message from Ben from 2005 where he suggests increasing the
blocksize and conn_bufsiz settings in Globals.py:
https://lists.gnu.org/archive/html/rdiff-backup-users/2005-10/msg00062.html
What he didn't say was whether this needed to be changed on the target
server, the backup server, or both. Nor do I know if that would actually
help this situation.
Do you have any ideas?
Thanks,
-derek
PS: According to rpm, both systems are running version 1.2.8.
--
Derek Atkins 617-623-3745
[email protected] www.ihtfp.com
Computer and Internet Security Consultant
_______________________________________________
rdiff-backup-users mailing list at [email protected]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
