On 9/14/06, Scott Atchley <[EMAIL PROTECTED]> wrote:
You do not mention what interconnect (protocol or advertised throughput) that you are considering. If you are considering gigabit Ethernet, you can skip the rest of this message.
All, Thank you very much for your input, comments, contacts, and suggestions. This information is greatly appreciated and will go a long way in helping us in overcoming the performance issues we currently face. We are currently running GigE as our interconnect. Basically we are currently running two NFS servers out to our web servers. We also are running three MySQL servers. The MySQL instances are segmented right now, but we are about to start an eval of Continuent's M//Cluster software. As stated, our FS infrasdtructure leaves much to be desired. The current setup involving NFS servers (Dell PE 2850 with local 1TB local storage 10K scsi disks) have not performed well. We are constantly IO waiting. Another interesting thing is, each MySQL server is using a ISCSI block device from SATAII NAS servers that we built using generic super micro boards and Areca controllers. Each of these boxes has approx 2.1TB of usable disk, and the performance has been suprisingly good. The Areca 1160 controllers with 1GB cache are handling the load, especially compared to our FS infrastructure of localized disks (I would have thought the opposite would be true), as the mysql disk IO pattern would be more smaller random IO, and the FS is mostly read (serving up web pages). We have made pretty much every last ounce of optimization we can on the NFS side (TCP, packet sizes, hugemem kernels, tried David Howells fscache on web client side) but non has been the silver bullet we've been looking for, which led us down the parallel fs path. We were hoping to ultimately employ something like the following: Take the ISCSI target boxes, the two nfs servers with local disk, and possibly purchase additional hardware if necessary, and run a parallel fs across them. GPFS looked the most feature packed (concurrent file access, multiple read/write, client side caching, PFS Data Striping, and storage pools). I think ultimately the way to describe our current setup is we out grown the sweet spot of the ad-hoc NFS, mysql approach we are currently employing, vs. not being large enough quite yet for the full on Cluster FS that we seek to employ.. Yet in an effort to scale to the sky, we are going to try to do this correctly, rather than continually being reactive. Best Regards, - Brent _______________________________________________ Beowulf mailing list, [email protected] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
