more situations for the -W flag
1)if either your client or server have a slow CPU. You should time a full
backup with and without the -W. Many times I find that the -W improves
backup times even over remote connections if you have a slow CPU in the
mix. I backup an older Alphaserver 1Ghz box and rsync pegs the CPU when
doing incremental work, the -W takes my backups from 1.5 hours to 20
minutes(100MB ethernet, 4GB per day)
2)whenever you are on a LAN, use the -W. Assuming your backups are at
night, you will not miss the bandwidth used up but should see a big
improvement. keep in mind that rsync still will only copy files that have a
size and timestamp change.
3)when you have a million files or so that are small on a LAN. Time the
backup, see if the CPU time is worth the wait. I have a host that has well
over 2 Million small files. rsync typically transfers these files at
<200KB/s on 100MB ethernet while it will do 3-4MB/s with the -W option. the
reason is that the rsync algorythm us a lot of IO transactions while the -W
is more sequential reads, lowering IO and drastically improving performance.
sometimes the advantages of rsync are unneccessary and are a hindrance.
Most of the time we need just a simple check for timestamp, size, and
ctime.
On Thu, Jul 3, 2008 at 4:13 PM, Tony Schreiner <[EMAIL PROTECTED]> wrote:
> Evren Yurtesen wrote:
> > Renke Brausse wrote:
> >> Hello Tony,
> >>
> >>> I've written before about backups involving very big files that
> >>> seem to execute slowly.
> >>>
> >>
> >>> What can be slowing things down so much? Except for this operation,
> >>> everything else runs about as I would expect.
> >>
> >> I have no clue what the reason is but I experienced that backups of
> >> large files are much faster with tar over ssh instead of rsync over ssh.
> >>
> >> Not an explanation but maybe this can solve your problem.
> >>
> >
> > I believe the reason for this is how rsync works. It normally tries to
> > transfer only the changed parts of the file. This is to save
> > bandwidth, to do this, it has to scan the whole file on both sides(I
> > guess). This is unnecessary unless you are over slow links. You might
> > want to try the whole-file option with rsync:
> >
> > -W, --whole-file copy files whole (w/o delta-xfer
> > algorithm)
> >
> > Please let us know the results, as a side-note if you still want to
> > shrink the transferred file size you can use the ssh compression with
> > -C option of ssh.
> >
> > Thanks,
> > Evren
> Thank you, I will try that. Might be a few days before I have anything
> to report.
> Tony
>
>
> -------------------------------------------------------------------------
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> _______________________________________________
> BackupPC-users mailing list
> [email protected]
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
BackupPC-users mailing list
[email protected]
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
