On 09/13/2012 11:21 AM, hol...@th.physik.uni-frankfurt.de wrote: >> I assume so, but just to be clear you witnessed this behavior even with >> the -I (directio) parameter? > > Yes.
Sorry for the confusion, this question was aimed at Joe, not you Andrew. I was wondering if Joe had seen caching effects even when using IOzone with the -I parameter. >> Can you rerun those tests with, 16 and 32 procs? I've run into some > > 1 proc Children see throughput for 1 random writers = 46036.32 > KB/sec > 2 proc Children see throughput for 2 random writers = 82828.13 > KB/sec > 4 proc Children see throughput for 4 random writers = 126709.65 > KB/sec > 8 proc Children see throughput for 8 random writers = 190070.96 > KB/sec > 16 proc Children see throughput for 16 random writers = 273970.94 > KB/sec > > 1 proc Children see throughput for 1 random readers = 109169.52 > KB/sec > 2 proc Children see throughput for 2 random readers = 202556.82 > KB/sec > 4 proc Children see throughput for 4 random readers = 381504.25 > KB/sec > 8 proc Children see throughput for 8 random readers = 719108.27 > KB/sec > 16 proc Children see throughput for 16 random readers = 1152648.13 > KB/sec Ah, this looks great! You added 50% IOPs with the doubling of procs, and I would bet you could squeeze a little more out by going to 24 or 32 procs. > I am quite sure I am not doing any local caching. Ok, great! But remote caching likely is still happening unless you blow away those files in between runs, so make sure you're doing that. Obviously this is harder for the reads, but if you have root permissions to the nexgenta gear just nuke the kernel buffer cache on that end. > Why is each process IO limited like that? Anytime a process is forced to wait or does so voluntarily, you are going to run into this type of limiting. By increasing the numbers of threads or processes you are able to "hide" some of this turn-around gap because another process that is available to run jumps right in and uses the bandwidth. My dissertation /should/ fix this such that a single process can get full bandwidth, but that's some 2 years and a bunch of sleepless nights away, so don't hold your breath, ;D. Best, ellis _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
