Hi, sorry for the delay... On Tue 02 May 2006, Toni Mueller wrote: > > I am experiencing a performance problem rsync'ing from one machine to > another in my LAN. The LAN is 100MBit/s Ethernet, idle, and the two > machines are also mostly idle, with rsync + ssh together using less than > 15% CPU on the slower machine, and some 3% on average on the faster > (receiving) machine. The drives are U160 or U320 in the sender machine, > and SATA in the other. Therefore, I expect the network to be saturated > by the transfer. But in reality, I get a varying transfer rate between > some 200KB/s and 7MB/s, with some 500KB/s on average (over hours, I'm > trying to transfer some 50 gigs). When I transfer large, already > compressed, files or ISOs, transfer is abysmally slow, and when I > transfer lots of small files, performance soars. I have RSYNC_RSH=ssh on > both ends, and transfer using rsync -av (note: not -u, not -z) in order > to minimize CPU usage. But the drive lights also don't indicate much > activity.
I've been pondering this problem, but I can't really explain it. At work I use rsync extensively (transferring 400MB ~ 1TB every day), and I see speeds of up to 70MB/s (over 1Gbit/s ethernet). This is between a mix of SCSI, SATA, SAS, Fibre Channel storage, so that doesn't seem to matter. Yesterday I transferred almost 4 million image (jpg) files from one system to another, average throughput was 8MB/s (bottleneck was disk IO in updating all those inodes / directories). So you seem to have no significant disk activity, and no significant CPU load... (Note that I prefer using rsync daemons over local networks, which eliminate the ssh overhead.) I am wondering whether perhaps there is some problem with your network, as packet loss will kill your throughput, especially when trying to transfer quickly. You could try some tests with netcat and buffer to see what raw throughput you can achieve. Check half/full duplex and flow control settings on your NICs and your switches. > And why is transfer over a WAN consistently fast enough to saturate my > WAN link(s), ie, faster than in my LAN (but using different data)? The roundtrip delay may prevent triggering whatever is wrong with the network...? In any case, I strongly suspect the problem is *not* with rsync; sorry... Paul Slootman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
