Tim Cutts wrote:
Optimising for throughput, at least with an embarrassingly parallel workload of serial jobs like we have here, is trivial; a single first-come-first-served queue is optimal, as long as the code is well written, and doesn't block too much on shared resources like file servers or databases.
but what if you have a bi-cpu bi-core machine to which you assign 4 slots. Now one slot is being used by a process which performs heavy IO. Suppose another process is launched that performs heavy IO. In that case the latter process should wait until the first one is done to avoid slowing down the efficiency of the system. Generally however, clusters take only time and memory requirements into account.
Additionally, in the case above, for optimising the efficiency of the node, I might prefer to launch just 1 process which uses 4 threads to perform multi-threaded (BLAS) calculations.
Most of the scheduling tweaks we put in to improve fairness and response time actually have a negative impact on throughput and efficiency.
Thanks. I totally agree but people find it hard to believe. Having a nice article explaining all this would make it easier to explain this (over and over again).
t _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
