-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/03/11 16:26, Mark Hahn wrote:
> if the app doesn't control this (or you with numactl), > then you should expect performance to lie somewhere > between the two extremes (fully local vs fully remote). > the kernel does make some effort at keeping things local > - and for that matter, avoiding moving a process among > multiple cores/sockets. It's worth also mentioning the issue of "NUMA diffusion" through swapping made by David Singleton from ANU on the hwloc-devel list: http://www.open-mpi.org/community/lists/hwloc-devel/2011/02/2012.php # Unless it has changed very recently, Linux swapin_readahead # is the main culprit in messing with NUMA locality on that # platform. Faulting a single page causes 8 or 16 or whatever # contiguous pages to be read from swap. An arbitrary contiguous # range of pages in swap may not even come from the same process # far less the same NUMA node. My understanding is that since # there is no NUMA info with the swap entry, the only policy # that can be applied to is that of the faulting vma in the # faulting process. The faulted page will have the desired NUMA # placement but possibly not the rest. So swapping mixes # different process' NUMA policies leading to a "NUMA diffusion # process". Keep in mind that the reason that ANU runs systems with swap is so they can suspend jobs, page the entire thing out and start a new higher priority job. Running without swap isn't really an option for them.. cheers, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2UYZYACgkQO2KABBYQAh/YxwCdFXz5yAdpKqz3G/Sk+AY73E2E XmMAn0aws1puNktvnaHQbhycul7pMrRp =jke/ -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
