Les Mikesell wrote: > Jeff Siddall wrote: >> Rob Owens wrote: >>> Kevin DeGraaf wrote: >>>> 2. Use the remote server to hold a copy of the on-site BackupPC >>>> server's file pool. The pool would be rsync'ed on-site initially and >>>> then rsync'ed remotely from then on. >>>> >>> Against all advice on this list, I did this yesterday. I rsync'd my >>> pool to a larger drive in the same machine. My pool size is 347G and >>> it's been running for the past 15 hours or so. I've transfered 334G so >>> far. My server has 3GB of RAM. It hasn't crashed, like many people >>> thought it might, and it has only used 2MB of swap. >>> >>> -Rob >> I never asked for anyone's opinion before trying this myself, but I have >> been [apparently] successfully rsyncing a copy of my BackupPC pool >> remotely over a DSL line for months now. I have yet to encounter >> issues, other than a few rsync warnings when the pool changes during the >> rsync -- which does take a few hours at DSL's 800 kbps uplink. The pool >> is about 120 GB, and changes by about 1 GB each day. >> >> I did my initial "remote" disk setup a bit differently. Instead of >> rsyncing the pool, I populated my "remote" drive by mounting it locally >> on the BackupPC server and doing a cp -a. No issues with that of >> course. Then I took the drive out of the BackupPC server and stuck it >> in the remote machine. From that point on rsync with the -H option has >> been keeping the pools in sync. >> >> Can anyone explain why this is a bad idea? > > Rsync tends to have a hard time with a backuppc archive because of the > sheer number of directory/file entries and also the number of hardlinks > which need a brute-force lookup to find the matching entries. Are you > using rsync 3.x? It may be that the recent changes have significantly > improved its ability to handle this scenario.
Nope, I'm using rsync-2.6.9-5.fc8 on relatively slow hardware. The PC is a dual core Athlon 2.1 GHz with 4 GB RAM, and the pool filesystem is on an external SATA drive running ext3. As this job runs in the middle of the night I have no idea how much RAM it is gobbling. Maybe I should look into that, although the file list size of 150 MB or so wouldn't indicate I should be running short of RAM. All that being said my pool is relatively small at 5.4 million files and 645 GB (as reported by rsync -- counting all copies of linked files I assume). I suspect (or perhaps hope) that rsync 3 may solve the known problems before my pool size becomes an issue. Here's a dump from the last log: Backup started Sun Oct 5 23:03:01 EDT 2008 Number of files: 5350429 Number of files transferred: 1691 Total file size: 692701780215 bytes Total transferred file size: 813298248 bytes Literal data: 806422416 bytes Matched data: 6876306 bytes File list size: 157784599 File list generation time: 3547.271 seconds File list transfer time: 0.000 seconds Total bytes sent: 1513518 Total bytes received: 965761770 sent 1513518 bytes received 965761770 bytes 53177.67 bytes/sec total size is 692701780215 speedup is 716.14 Backup finished Mon Oct 6 04:13:33 EDT 2008 It took about 5 hours to sync the 120 GB or so, one hour of which was building the file list. Maybe more files (possibly more data) would break things but so far it seems to be running fine. Jeff ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ BackupPC-users mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
