Dear Arun,
thank you for checking the workaround scripts.
I've modified detectCores() to use GetLogicalProcessorInformationEx. It
is in revision 75198 of R-devel, could you please test it on your
machines? For a binary, you can wait until the R-devel snapshot build
gets to at least this svn revision.
Thanks for the link to the processor groups documentation. I don't have
a machine to test this on, but I would hope that snow clusters (e.g.
PSOCK) should work fine on systems with >64 logical processors as they
spawn new processes (not just threads). Note that FORK clusters are not
supported on Windows.
Thanks
Tomas
On 08/21/2018 02:53 PM, Srinivasan, Arunkumar wrote:
Dear Tomas, thank you for looking into this. Here's the output:
# number of logical processors - what detectCores() should return
out <- system("wmic cpu get numberoflogicalprocessors", intern=TRUE)
[1] "NumberOfLogicalProcessors \r" "22 \r" "22
\r"
[4] "20 \r" "22 \r" "\r"
sum(as.numeric(gsub("([0-9]+).*", "\\1", grep("[0-9]+[ \t]*", out,
value=TRUE))))
# [1] 86
[I've asked the IT team to understand why one of the values is 20 instead of
22].
# number of cores - what detectCores(FALSE) should return
out <- system("wmic cpu get numberofcores", intern=TRUE)
[1] "NumberOfCores \r" "22 \r" "22 \r" "20 \r"
"22 \r"
[6] "\r"
sum(as.numeric(gsub("([0-9]+).*", "\\1", grep("[0-9]+[ \t]*", out,
value=TRUE))))
# [1] 86
[Currently hyperthreading is disabled. So this output being identical to the
previous output makes sense].
system("wmic computersystem get numberofprocessors")
NumberOfProcessors
4
In addition, I'd also bring to your attention this documentation:
https://docs.microsoft.com/en-us/windows/desktop/ProcThread/processor-groups on
processor groups which explain how one should go about running a process ro run
on multiple groups (which seems to be different to NUMA). All this seems overly
complicated to allow a process to use all cores by default TBH.
Here's a project on Github 'fio' where the issue of running a process on more
than 1 processor group has come up - https://github.com/axboe/fio/issues/527
and is addressed -
https://github.com/axboe/fio/blob/c479640d6208236744f0562b1e79535eec290e2b/os/os-windows-7.h
. I am not sure though if this is entirely relevant since we would be forking
new processes in R instead of allowing a single process to use all cores.
Apologies if this is utterly irrelevant.
Thank you,
Arun.
From: Tomas Kalibera <tomas.kalib...@gmail.com>
Sent: 21 August 2018 11:50
To: Srinivasan, Arunkumar <arunkumar.sriniva...@uk.mlp.com>;
r-devel@r-project.org
Subject: Re: [Rd] Get Logical processor count correctly whether NUMA is enabled
or disabled
Dear Arun,
thank you for the report. I agree with the analysis, detectCores() will only
report logical processors in the NUMA group in which R is running. I don't have
a system to test on, could you please check these workarounds for me on your
systems?
# number of logical processors - what detectCores() should return
out <- system("wmic cpu get numberoflogicalprocessors", intern=TRUE)
sum(as.numeric(gsub("([0-9]+).*", "\\1", grep("[0-9]+[ \t]*", out,
value=TRUE))))
# number of cores - what detectCores(FALSE) should return
out <- system("wmic cpu get numberofcores", intern=TRUE)
sum(as.numeric(gsub("([0-9]+).*", "\\1", grep("[0-9]+[ \t]*", out,
value=TRUE))))
# number of physical processors - as a sanity check
system("wmic computersystem get numberofprocessors")
Thanks,
Tomas
On 08/17/2018 05:11 PM, Srinivasan, Arunkumar wrote:
Dear R-devel list,
R's detectCores() function internally calls "ncpus" function to get the total
number of logical processors. However, this doesnot seem to take NUMA into account on
Windows machines.
On a machine having 48 processors (24 cores) in total and windows server 2012
installed, if NUMA is enabled and has 2 nodes (node 0 and node 1 each having 24
CPUs), then R's detectCores() only detects 24 instead of the total 48. If NUMA
is disabled, detectCores() returns 48.
Similarly, on a machine with 88 cores (176 processors) and windows server 2012,
detectCores() with NUMA disabled only returns the maximum value of 64. If NUMA
is enabled with 4 nodes (44 processors each), then detectCores() will only
return 44. This is particularly limiting since we cannot get to use all
processors by enabling/disabling NUMA in this case.
We think this is because R's ncpus.c file uses "PSYSTEM_LOGICAL_PROCESSOR_INFORMATION"
(https://msdn.microsoft.com/en-us/library/windows/desktop/ms683194(v=vs.85).aspx) instead of
"PSYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX"
(https://msdn.microsoft.com/en-us/library/windows/desktop/dd405488(v=vs.85).aspx). Specifically,
quoting from the first link:
"On systems with more than 64 logical processors, the GetLogicalProcessorInformation
function retrieves logical processor information about processors in the
https://msdn.microsoft.com/en-us/library/windows/desktop/dd405503(v=vs.85).aspx to which
the calling thread is currently assigned. Use the
https://msdn.microsoft.com/en-us/library/windows/desktop/dd405488(v=vs.85).aspx function
to retrieve information about processors in all processor groups on the system."
Therefore, it might be possible to get the right count of total processors even with NUMA
enabled by using "GetLogicalProcessorInformationEX". It'd be nice to know what
you think.
Thank you very much,
Arun.
--
Arun Srinivasan
Analyst, Millennium Management LLC
50 Berkeley Street | London, W1J 8HD
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
