On Fri, 30 Nov 2007, Mark Kozikowski wrote:
Hello all, I have been reading the list for quite a while and have gleaned a boat load of information. Thank you. Now I am going to ask help for a problem I am having. I have simulation program that runs on between 5 and 15 nodes. This depends on the desired fidelity of the simulation. When I approach the higher fidelity levels, the simulation starts to choke on the quantity of data being processed. It appears that the system is failing on I/O. Transferring large amounts of time critical data between process elements. I a running on a mostly standard Red Hat distro, no special compiling or running architectures are in place. Do any of you have suggestions as to how I might start getting control of this I/O problem?
Sure. Start by describing your cluster IN DETAIL. What kind of systems, how many processors, how much memory and disk, and above all, what kind of network(s) and what speed. This will help you figure out if there are any relatively "simple" solutions, like getting more/better network bandwidth. The second aspect of fixing your problem is to analyze it mathematically and logically fairly completely. Supposing that you've got all the bandwidth between hosts that you can reasonably afford, the next question is whether or not you can reorganize your task to take maximal advantage of that bandwidth. This may involve significantly altering your task organization in possibly non-intuitive ways, or it may involve organizing your computation so data transfers themselves occur in a maximally parallel way. Whether or not this helps you increase your throughput depends in some measure on how your task is organized and if there is room for improvement. rgb
Thanks for any help. Mark Kozikowski Advanced Acoustic Concepts _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
-- Robert G. Brown Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone(cell): 1-919-280-8443 Web: http://www.phy.duke.edu/~rgb Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977 _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
