On 09/14/2017 11:34 AM, Faraz Hussain wrote:
Earlier I had posted about one of our blades running 30-50% slower
than other ones despite having identical hardware and OS. I followed
the suggestions and compared cpu temperature, memory, dmesg and
sysctl. Everything looks the same.
I then used "perf stat" to compare speed of pigz ( parralel gzip ).
The results are quite interesting. Using one cpu, the slow blade is as
fast as the rest! But as I use more cpus, the speed decreases linearly
from 3.1Ghz to 0.4 Ghz. See snippets from "perf stat" command below.
All tests were on /tmp to eliminate any nfs issue. And same behavior
is observed with any multi-threaded program.
What does numastat report? /tmp is a ramdisk or tmpfs? Are the
nodes/cpus otherwise idle? What does lscpu on a good/bad node report?
If it decreases on a 1/Ncpu curve, then you have a fixed sized resource
bandwidth contention issue you are fighting. The question is what.
--
Joe Landman
e: joe.land...@gmail.com
t: @hpcjoe
w: https://scalability.org
g: https://github.com/joelandman
l: https://www.linkedin.com/in/joelandman
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
