Bernard Li wrote:
I get the sense that CPU affinity is beneficial even for
embarrassingly/serial jobs -- however I am curious whether anybody has
Well, yes it is.
actual numbers to back this? And is the potential benefits worth the
Here are some, more about process CPU->memory affinity than process
CPU->cache affinity...
I am forcing the memory to be on a different CPU than running the code:
[EMAIL PROTECTED]:/big/stream-jb-2006-6-8$ numactl --physcpubind=1
--membind=1 ./stream_d_c_omp_x86_64
Function Rate (MB/s) RMS time Min time Max time
Copy: 3122.6206 0.2050 0.2050 0.2052
Scale: 3107.5644 0.2061 0.2059 0.2065
Add: 3118.4948 0.3080 0.3078 0.3084
Triad: 3107.2203 0.3091 0.3090 0.3093
I am forcing the memory to be on the same CPU running the code:
[EMAIL PROTECTED]:/big/stream-jb-2006-6-8$ numactl --physcpubind=1
--membind=0 ./stream_d_c_omp_x86_64
Function Rate (MB/s) RMS time Min time Max time
Copy: 3705.1849 0.1729 0.1727 0.1734
Scale: 3954.8385 0.1619 0.1618 0.1623
Add: 3893.1524 0.2468 0.2466 0.2472
Triad: 3857.3264 0.2490 0.2489 0.2493
This is more of a memory affinity issue, you want your code running on
the memory local to the particular CPU. A decade ago, we saw all sorts
of performance degredation when processes would migrate from CPU to CPU
on the big SGI machines, partially defeating the utility of cache (not
to mention forcing a huge amount of inter-cpu traffic when you had
massive invalidation storms as a result of the scheduler moving the
process).
Simply having the scheduler leave the process on the same CPU turned out
to be a significant win. Especially for long running jobs that were
very cache intensive (e.g. such as EP codes, Monte Carlo, ...)
time/effort to set this up rather than let the default Linux scheduler
deal with it.
The linux scheduler is actually quite reasonable about this (these days,
with a modern kernel). Some older kernels had problems.
Cheers,
Bernard
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: [EMAIL PROTECTED]
web : http://www.scalableinformatics.com
http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
