On Thu, 17 Jul 2003, Reuben D. Budiardja wrote: > On Thursday 17 July 2003 12:47 pm, Bret Hughes wrote: > > On Thu, 2003-07-17 at 08:27, Reuben D. Budiardja wrote: > > > Hello, > > > I am wondering if anyone can help me with this scenario. > > > > > > Suppose I want to copy large data set over the net (about 20 GB). it's a > > > bunch of files with directories and subdirectories, so basically I'm > > > trying to copy a directory tree. > > > Now, I don't want to copy the whole thing in one sit since it's large. I > > > want to do it incrementally during off-peak hours, so, say everyday > > > between 2-3 AM I would copy 500 MB, and then the next day continue with > > > another ~500MB, and the next day, etc, etc, until it's completed. The > > > content of source directory will change, although not by much at all, so > > > this needs to be able to keep trakc of that and sync with the changes. > > > > > > I have a shell access (SSH) on both machines, the source machine, and the > > > target machine. My local machine is the target machine. So I think > > > something like 'rsync' should probably work, combine with something else > > > for scheduling (cron maybe). But I am not sure how to stop after the > > > first ~500 MB is transferred, and continue the next day (of course the > > > number 500MB does not have to be exact, it could be a rough rounding, > > > because it's probably easier to stop after one file is completed, than > > > stop in the middle of the file). > > > > I would set up individual jobs of the approximate size and use rsync > > (over ssh of course) on them for the initial runs and then run rsysnc > > daily to keep them in sync. > > But that would require me to manually calculate the size of directories (eg. > with 'du'). It would be tedious since it's a lot of directories and sub-dir. > I am wondering if there's a way to automate the process. > > Thanks though. > RDB
Perhaps you could tar-gzip the whole bunch, then use the "split" command to chop them into bite-size pieces. Regards, Mike Anderson > > > > > -- redhat-list mailing list unsubscribe mailto:[EMAIL PROTECTED] https://www.redhat.com/mailman/listinfo/redhat-list
