David, Thanks for the pointer. This was it. I'm now able to sequentially read at line speed ( 1000Mb/s ) over NFS. Also, thanks for mentioning it could be changed without a reboot. In RHEL4 I was passing it as a kernel option and it wasn't available under /sys, making testing difficult. In RHEL5 I can it access under /sys which made it a lot easier to test. I wouldn't have bothered to look if you hadn't mentioned it.
On Fri, 2007-08-24 at 09:08 -0700, David Mathog wrote: > > Glen Dosey <[EMAIL PROTECTED]> wrote > > On Fri, 2007-08-24 at 08:45 -0400, Lawrence Sorrillo wrote: > > > I am uncertain what this step does.... > > > > > > now we unmount the NFS share, recreate the file on the server, and > remount > > > it to clear the client cache but leave it cached on the server > > > > > > [EMAIL PROTECTED] ~]# dd if=/mnt/array3/file.dd of=/dev/null bs=4k > > > 524287+0 records in > > > 524287+0 records out > > > 2147479552 bytes (2.1 GB) copied, 18.5161 seconds, 116 MB/s > > > > > > > > > What does it accomplish? > > > > I was using this step to illustrate that NFS itself is capable of > > transferring data across the GigE network at near wires speeds. > > > > By unmounting the NFS share we clear the file from being cached in RAM > > on the client, ensuring that it must be gotten from the server again via > > NFS as opposed to being grabbed from RAM on the client. > > I ran into a nasty bug involving "dump" on recent kernels, > the long and still not fully resolved story is here: > > http://bugzilla.kernel.org/show_bug.cgi?id=8636 > > In a nutshell, dump on recent kernels runs hideously slow on (P)ATA > disks (by a factor of 6!) because of this change in the .config file: > > -CONFIG_DEFAULT_IOSCHED="cfq" > +CONFIG_DEFAULT_IOSCHED="anticipatory" > > (cfq in the new kernels, anticipatory in the old ones). The "fix" was: > > echo "anticipatory" > /sys/block/hda/queue/scheduler > > Now this may be completely irrelevant, for instance, there was no > problem with the SCSI disks I tested, but there might be with your raid > controller. If NFS can move the data across the network at full speed, > but can't seem to read nearly as fast as the controller, it's possible > that the scheduler you're using is involved. There are 4 to choose > from, perhaps give them all a test and see if any of them improve matters? > > Regards, > > David Mathog > [EMAIL PROTECTED] > Manager, Sequence Analysis Facility, Biology Division, Caltech > _______________________________________________ > Beowulf mailing list, [email protected] > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, [email protected] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
