[Rd] Can't use custom package on windows 64-bit
I've been developing a package to use S3 bucket on AWS by using libs3 code. I have two problems. The first is, by standard, it will attempt to install i386 and x64 if I don't have a configure.win. The problem with this is that while everything appears to compile correctly, I get this error when trying to load the x64 library: "LoadLibrary error: %1 is not a valid Win32 application" I am able to load the 32-bit library, but when I try to load that in Rstudio, it says: "package is not installed for 'arch=x64'". Now I know that I can simply switch Rstudio to point to 32-bit R, but that is not ideal. Is there a way to easily fix that? So that I can either compile correctly the 64-bit, or I can run the 32-bit in 64-bit R. Here's my source code: https://github.com/Gastrograph/RS3 Thank you! -- Evan Farrell *Application Engineer* *Analytical Flavor Systems, LLC. * Mobile: (724) 831 - 9537 www.gastrograph.com All materials confidential If you are not the intended recipient, do not read (too late?), copy or distribute this e-mail or any attachments.Instead, please notify the sender and delete the e-mail and any attachments. Thank you. Copyright © Analytical Flavor Systems 2014 [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Can't use custom package on windows 64-bit
On 6 August 2014 at 09:47, Evan Farrell wrote: | I've been developing a package to use S3 bucket on AWS by using libs3 code. | I have two problems. The first is, by standard, it will attempt to | install i386 and x64 if I don't have a configure.win. | | The problem with this is that while everything appears to compile | correctly, I get this error when trying to load the x64 library: | | "LoadLibrary error: %1 is not a valid Win32 application" | | I am able to load the 32-bit library, but when I try to load that in | Rstudio, it says: "package is not installed for 'arch=x64'". Now I know | that I can simply switch Rstudio to point to 32-bit R, but that is not | ideal. | | | Is there a way to easily fix that? So that I can either compile correctly | the 64-bit, or I can run the 32-bit in 64-bit R. | | Here's my source code: https://github.com/Gastrograph/RS3 Thanks for that pointer. Could you consider "just" using Linux for now? Doing Windows is entirely doable, but more involved. As I recall, you need tricks such as not using a configure.win in order to build with both default arches on Windows. There are packages that do this, and you can "borrow" solutions from them. But none springs to my mind rightaway as I tend to deploy on Linux first, or else work with the very kind CRAN folks and have them provide 32 and 64 bit libraries on the builders themselves (as eg recently with hiredis) :) Medium-term you could do this too as libS3 is open source. Until then you can of course emulate that scheme perfectly well by building the 32 and 64 bit DLLs, store them in your repo and have Makevars.win point to them via the same env var trick used at CRAN. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Subscripting Matrices
There seems to be a result type difference when subscripting a 6 x 1 matrix as compared to a 3 x 2 matrix that is caused by the ncol = 1 compared to ncol > 1. > ThinMatrix <- matrix(1:6,ncol=1) > ThinMatrix [,1] [1,]1 [2,]2 [3,]3 [4,]4 [5,]5 [6,]6 > FatMatrix <- matrix(1:6,ncol=2) > FatMatrix [,1] [,2] [1,]14 [2,]25 [3,]36 > dim(ThinMatrix[TRUE,]) NULL #Though this value should be 6 1 > dim(FatMatrix[TRUE,]) [1] 3 2 Thanks for your help. Terry Ireland __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Subscripting Matrices
You want `drop=FALSE`: > dim(ThinMatrix[TRUE, , drop=FALSE]) [1] 6 1 >From ?"[": drop: For matrices and arrays. If ‘TRUE’ the result is coerced to the lowest possible dimension (see the examples). This only works for extracting elements, not for the replacement. See ‘drop’ for further details. And R inferno 8.1.44: http://www.burns-stat.com/pages/Tutor/R_inferno.pdf Gabor On Wed, Aug 6, 2014 at 11:07 AM, Terrence Ireland wrote: > There seems to be a result type difference when subscripting a 6 x 1 matrix > as compared to a 3 x 2 matrix that is caused by the ncol = 1 compared to > ncol > 1. > >> ThinMatrix <- matrix(1:6,ncol=1) >> ThinMatrix > [,1] > [1,]1 > [2,]2 > [3,]3 > [4,]4 > [5,]5 > [6,]6 >> FatMatrix <- matrix(1:6,ncol=2) >> FatMatrix > [,1] [,2] > [1,]14 > [2,]25 > [3,]36 >> dim(ThinMatrix[TRUE,]) > NULL #Though this value should be 6 1 >> dim(FatMatrix[TRUE,]) > [1] 3 2 > > Thanks for your help. > > Terry Ireland > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Subscripting Matrices
On Aug 6, 2014, at 10:07 AM, Terrence Ireland wrote: > There seems to be a result type difference when subscripting a 6 x 1 matrix > as compared to a 3 x 2 matrix that is caused by the ncol = 1 compared to ncol > > 1. > > > ThinMatrix <- matrix(1:6,ncol=1) > > ThinMatrix > [,1] > [1,]1 > [2,]2 > [3,]3 > [4,]4 > [5,]5 > [6,]6 > > FatMatrix <- matrix(1:6,ncol=2) > > FatMatrix > [,1] [,2] > [1,]14 > [2,]25 > [3,]36 > > dim(ThinMatrix[TRUE,]) > NULL #Though this value should be 6 1 > > dim(FatMatrix[TRUE,]) > [1] 3 2 > > Thanks for your help. > > Terry Ireland Hi, This question really should have gone to R-Help, not R-Devel and it is a FAQ: http://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-do-my-matrices-lose-dimensions_003f > str(ThinMatrix[TRUE,]) int [1:6] 1 2 3 4 5 6 Regards, Marc Schwartz __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Can't use custom package on windows 64-bit
On 06/08/2014 14:47, Evan Farrell wrote: I've been developing a package to use S3 bucket on AWS by using libs3 code. I have two problems. The first is, by standard, it will attempt to install i386 and x64 if I don't have a configure.win. Or if you use --force-biarch or --merge-multiarch: see ?INSTALL and 'Writing R Extensions'. The problem with this is that while everything appears to compile correctly, I get this error when trying to load the x64 library: "LoadLibrary error: %1 is not a valid Win32 application" I am able to load the 32-bit library, but when I try to load that in Rstudio, it says: "package is not installed for 'arch=x64'". Now I know that I can simply switch Rstudio to point to 32-bit R, but that is not ideal. That' s confusing error message from Windows. It most likely means that some DLL which your package's DLL is trying to link to cannot be found, or can be found but is 32-bit. (In theory at least a 64-bit DLL of the required name will be found first, if there is one.) Is there a way to easily fix that? So that I can either compile correctly the 64-bit, or I can run the 32-bit in 64-bit R. You cannot load 32-bit DLLs into a 64-bit process. (Here that means the process linking to the R DLL: RStudio is just an arm's-length front end that communicates with a client in a separate process.) Here's my source code: https://github.com/Gastrograph/RS3 You copy DLLs into the package from elsewhere, and likely they are the problem. If you call dyn.load() on each of those you may get a more informative error message. Using 'Dependency Walker' (see Writing R Extensions) can sometimes be revealing as to which DLL is missing/corrupt. Thank you! __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] portableParalleSeeds Package violation, CRAN exception?
I'm writing to ask for a policy exception, or advice on how to make
this package CRAN allowable.
http://rweb.quant.ku.edu/kran/src/contrib/portableParallelSeeds_0.9.tar.gz
Yesterday I tried to submit a package on CRAN and Dr Ripley pointed
out that I had not understood the instructions about packages. Here's
the part where the R check gives a Note
* checking R code for possible problems ... NOTE
Found the following assignments to the global environment:
File ‘portableParallelSeeds/R/initPortableStreams.R’:
assign("currentStream", n, envir = .GlobalEnv)
assign("currentStates", curStates, envir = .GlobalEnv)
assign("currentStream", 1L, envir = .GlobalEnv)
assign("startStates", runSeeds, envir = .GlobalEnv)
assign("currentStates", runSeeds, envir = .GlobalEnv)
assign("currentStream", as.integer(currentStream), envir = .GlobalEnv)
assign("startStates", runSeeds, envir = .GlobalEnv)
assign("currentStates", runSeeds, envir = .GlobalEnv)
Altering the user's environment requires a special arrangement with
CRAN. I believe this is justified, I'll sketch the reasons now. But,
mostly, I'm at your mercy and if there is any way to make this
possible, I would be very grateful.
To control & replace random number streams, it really is necessary to
alter the workspace. That's where the random generator state is
stored. It is acknowledged in Robert Gentleman' s Book, R Programming
for Bionformatics "The decision to have these [random generator]
functions manipulate a global variable, .Random.seed, is slightly
unfortunate as it makes it somewhat more difficult to manage several
different random number streams simultaneously” (Gentleman, 2009, p.
201).
I have developed an understandable set of wrapper functions that handle this.
Some of you may recall this project. I've asked about it here a couple
of times. We allow separate streams of randoms for different purposes
within a single R run. There is a framework to save 1000s of those
sets in a file, so it can be used on a cluster or in a single
workstation. This is handy because, when 1 run in 10,000 on the
cluster exhibits some weird behavior, we can easily re-initiate that
interactively and see what's going on.
I have a vignette "pps" that explains. I dropped a copy of that here
in case you don't want to get the package:
http://pj.freefaculty.org/scraps/pps.pdf
While working on that, I gained a considerably deeper understanding of
random generators and seeds. That is what this vignette is about
http://pj.freefaculty.org/scraps/PRNG-basics.pdf
We've been running simulations on our cluster with the
portableParallelSeeds framework for 2 years, we've never had any
trouble. We are able to re-start runs, verify random number draws in
separate streams.
PJ
--
Paul E. Johnson
Professor, Political Science Assoc. Director
1541 Lilac Lane, Room 504 Center for Research Methods
University of Kansas University of Kansas
http://pj.freefaculty.org http://quant.ku.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] More than one package document with the same name
- Original Message -
> From: "John McKown"
> To: "Jerry Davison"
> Cc: r-devel@r-project.org
> Sent: Tuesday, August 5, 2014 5:10:28 PM
> Subject: Re: [Rd] More than one package document with the same name
>
> On Tue, Aug 5, 2014 at 5:47 PM, Davison, Jerry
> wrote:
> > Hi,
> >
> > I sent this to the Bioconductor mailing list (q.v.), replies
> > recommended this forum. Here it is:
> > A suggestion. Typically a package provides documentation of two
> > types, one or more vignettes and a reference manual; and they have
> > the same file name, PackageName.pdf. When downloaded some file
> > name manipulation is required, or accept PackageName(1).pdf.
> >
> > I suggest the documents be given different names, for example the
> > reference manual could be named PackageNameRefMan.pdf. Package
> > checking could enforce the rule.
> > Jerry
> >
>
> Sounds like you want to put all the pdf files in a single directory.
Actually I think the issue (and Jerry can correct me if I'm wrong) is that
sometimes users want to download the documentation for a package from the
package landing page on Bioconductor or CRAN (without installing the package).
If both files are called pkg.pdf, then your web browser wants to rename the
second one to (e.g.) pkg(1).pdf (different browsers handle this differently).
Also, the two names on their own don't give you a clue as to which file is
which.
My own suggestion would be that the manual be renamed to pkg-manual.pdf. That
places no restrictions on vignette file names and makes it clear which file is
the manual.
Dan
> I
> like that too. I cheat. I run Linux. I have a ~/Documents/R-pdfs in
> which I keep "symlinks" to all the pdf files. And I do it similar to
> the way that you indicate. I make the name of the pdf be
> ${enclosing_directory}_${original_pdf_name.pdf}. I do something like:
>
> cd ~/Documents/R-pdf
> find /usr/lib64/R -name '*.pdf'|\
> while read i;do
> file=${i##*/};
> dir=${i%%/doc/*.pdf};
> package=${dir##*/};
> ln -s "$i" "${package}_${file}";
> done
>
> Above won't work if anything has a blank in it. Windows people tend
> to
> do this. Most UNIX people are better trained.
>
> It would be nice if the packagers did this. But the above works for
> me. On Linux and other UNIX like systems. Won't work for the poor,
> benighted Windows people. But I think something similar is possible.
> But I don't know Windows well enough.
>
> Maranatha! <><
> John McKown
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] portableParalleSeeds Package violation, CRAN exception?
Why not place them in the package environment? Gabor
On Wed, Aug 6, 2014 at 2:10 PM, Paul Johnson wrote:
> I'm writing to ask for a policy exception, or advice on how to make
> this package CRAN allowable.
>
> http://rweb.quant.ku.edu/kran/src/contrib/portableParallelSeeds_0.9.tar.gz
>
> Yesterday I tried to submit a package on CRAN and Dr Ripley pointed
> out that I had not understood the instructions about packages. Here's
> the part where the R check gives a Note
>
> * checking R code for possible problems ... NOTE
> Found the following assignments to the global environment:
> File ‘portableParallelSeeds/R/initPortableStreams.R’:
>assign("currentStream", n, envir = .GlobalEnv)
>assign("currentStates", curStates, envir = .GlobalEnv)
>assign("currentStream", 1L, envir = .GlobalEnv)
>assign("startStates", runSeeds, envir = .GlobalEnv)
>assign("currentStates", runSeeds, envir = .GlobalEnv)
>assign("currentStream", as.integer(currentStream), envir = .GlobalEnv)
>assign("startStates", runSeeds, envir = .GlobalEnv)
>assign("currentStates", runSeeds, envir = .GlobalEnv)
>
> Altering the user's environment requires a special arrangement with
> CRAN. I believe this is justified, I'll sketch the reasons now. But,
> mostly, I'm at your mercy and if there is any way to make this
> possible, I would be very grateful.
>
> To control & replace random number streams, it really is necessary to
> alter the workspace. That's where the random generator state is
> stored. It is acknowledged in Robert Gentleman' s Book, R Programming
> for Bionformatics "The decision to have these [random generator]
> functions manipulate a global variable, .Random.seed, is slightly
> unfortunate as it makes it somewhat more difficult to manage several
> different random number streams simultaneously” (Gentleman, 2009, p.
> 201).
>
> I have developed an understandable set of wrapper functions that handle this.
>
> Some of you may recall this project. I've asked about it here a couple
> of times. We allow separate streams of randoms for different purposes
> within a single R run. There is a framework to save 1000s of those
> sets in a file, so it can be used on a cluster or in a single
> workstation. This is handy because, when 1 run in 10,000 on the
> cluster exhibits some weird behavior, we can easily re-initiate that
> interactively and see what's going on.
>
> I have a vignette "pps" that explains. I dropped a copy of that here
> in case you don't want to get the package:
>
> http://pj.freefaculty.org/scraps/pps.pdf
>
> While working on that, I gained a considerably deeper understanding of
> random generators and seeds. That is what this vignette is about
>
> http://pj.freefaculty.org/scraps/PRNG-basics.pdf
>
>
> We've been running simulations on our cluster with the
> portableParallelSeeds framework for 2 years, we've never had any
> trouble. We are able to re-start runs, verify random number draws in
> separate streams.
>
> PJ
> --
> Paul E. Johnson
> Professor, Political Science Assoc. Director
> 1541 Lilac Lane, Room 504 Center for Research Methods
> University of Kansas University of Kansas
> http://pj.freefaculty.org http://quant.ku.edu
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] portableParalleSeeds Package violation, CRAN exception?
You can make an environment called streamsEnv in your package by adding
streamsEnv <- new.env()
to one of your R/*.R files. (its parent environment will be
namespace:yourPackage) and your functions can assign things to this
environment instead of to .GlobalEnv.
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Wed, Aug 6, 2014 at 11:10 AM, Paul Johnson wrote:
> I'm writing to ask for a policy exception, or advice on how to make
> this package CRAN allowable.
>
> http://rweb.quant.ku.edu/kran/src/contrib/portableParallelSeeds_0.9.tar.gz
>
> Yesterday I tried to submit a package on CRAN and Dr Ripley pointed
> out that I had not understood the instructions about packages. Here's
> the part where the R check gives a Note
>
> * checking R code for possible problems ... NOTE
> Found the following assignments to the global environment:
> File ‘portableParallelSeeds/R/initPortableStreams.R’:
>assign("currentStream", n, envir = .GlobalEnv)
>assign("currentStates", curStates, envir = .GlobalEnv)
>assign("currentStream", 1L, envir = .GlobalEnv)
>assign("startStates", runSeeds, envir = .GlobalEnv)
>assign("currentStates", runSeeds, envir = .GlobalEnv)
>assign("currentStream", as.integer(currentStream), envir = .GlobalEnv)
>assign("startStates", runSeeds, envir = .GlobalEnv)
>assign("currentStates", runSeeds, envir = .GlobalEnv)
>
> Altering the user's environment requires a special arrangement with
> CRAN. I believe this is justified, I'll sketch the reasons now. But,
> mostly, I'm at your mercy and if there is any way to make this
> possible, I would be very grateful.
>
> To control & replace random number streams, it really is necessary to
> alter the workspace. That's where the random generator state is
> stored. It is acknowledged in Robert Gentleman' s Book, R Programming
> for Bionformatics "The decision to have these [random generator]
> functions manipulate a global variable, .Random.seed, is slightly
> unfortunate as it makes it somewhat more difficult to manage several
> different random number streams simultaneously” (Gentleman, 2009, p.
> 201).
>
> I have developed an understandable set of wrapper functions that handle this.
>
> Some of you may recall this project. I've asked about it here a couple
> of times. We allow separate streams of randoms for different purposes
> within a single R run. There is a framework to save 1000s of those
> sets in a file, so it can be used on a cluster or in a single
> workstation. This is handy because, when 1 run in 10,000 on the
> cluster exhibits some weird behavior, we can easily re-initiate that
> interactively and see what's going on.
>
> I have a vignette "pps" that explains. I dropped a copy of that here
> in case you don't want to get the package:
>
> http://pj.freefaculty.org/scraps/pps.pdf
>
> While working on that, I gained a considerably deeper understanding of
> random generators and seeds. That is what this vignette is about
>
> http://pj.freefaculty.org/scraps/PRNG-basics.pdf
>
>
> We've been running simulations on our cluster with the
> portableParallelSeeds framework for 2 years, we've never had any
> trouble. We are able to re-start runs, verify random number draws in
> separate streams.
>
> PJ
> --
> Paul E. Johnson
> Professor, Political Science Assoc. Director
> 1541 Lilac Lane, Room 504 Center for Research Methods
> University of Kansas University of Kansas
> http://pj.freefaculty.org http://quant.ku.edu
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R process (and forked children via system2) are limited to 1 core?
Hi,
(Using R 3.1.1 on Ubuntu 12.04.4 LTS)
What is the recommended way for R to fork a (non-R) process that is not CPU
limited?
Currently I am using R's system2() call, and this is inheriting the environment
of the R process.
I notice that (at least on Linux) when I am poking around /proc that the R
process itself is setting up cpu limitations for itself (max 1 core).
Using strace, I see the following:
(strace output)
out.20612:sched_setaffinity(0, 128, {100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0}) = 0
And proc shows:
(cat /proc/nnn/status)
Cpus_allowed: 0001
Cpus_allowed_list: 0
See that the Cpus_allowed bitmask is a single core. Normally it's fff...f.
I want my child process (java in this case) not to share this limitation. What
is the recommended way of doing this from R?
Any ideas or suggestions appreciated!
Thanks,
Tom
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R process (and forked children via system2) are limited to 1 core?
On Wed, Aug 6, 2014 at 6:01 AM, Tom Kraljevic wrote:
>
> Hi,
>
>
> (Using R 3.1.1 on Ubuntu 12.04.4 LTS)
>
>
> What is the recommended way for R to fork a (non-R) process that is not CPU
> limited?
> Currently I am using R's system2() call, and this is inheriting the
> environment of the R process.
>
>
> I notice that (at least on Linux) when I am poking around /proc that the R
> process itself is setting up cpu limitations for itself (max 1 core).
>
>
> Using strace, I see the following:
>
> (strace output)
> out.20612:sched_setaffinity(0, 128, {100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> 0, 0, 0}) = 0
>
>
> And proc shows:
>
> (cat /proc/nnn/status)
> Cpus_allowed: 0001
> Cpus_allowed_list: 0
>
>
> See that the Cpus_allowed bitmask is a single core. Normally it's fff...f.
When I run R I see:
Cpus_allowed:ff
Cpus_allowed_list:0-7
It's possible (likely?) that the culprit here isn't R but rather some
other library that R is loading. Are you using OpenBLAS? By default
OpenBLAS will set an obnoxious cpu mask, unless you override this
using some obscure build system settings. (There might be a runtime
option for disabling it too, I don't remember. Note also that this is
just one of several obnoxious things OpenBLAS does unless you override
a bunch of obscure build system defaults -- building OpenBLAS
correctly is highly non-trivial.)
-n
--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R process (and forked children via system2) are limited to 1 core?
Hi Nathaniel,
Thanks for the suggestion.
Im actually not really using R to do any real work.
Im starting the R H2O package (a Java machine learning package) and forwarding
all the work to H2O.
This is the list of packages I have in R:
> search()
[1] ".GlobalEnv""package:h2o" "package:tools"
[4] "package:statmod" "package:rjson" "package:RCurl"
[7] "package:bitops""package:stats" "package:graphics"
[10] "package:grDevices" "package:utils" "package:datasets"
[13] "package:methods" "Autoloads" "package:base"
And here is the /proc info
tomk@mr-0xb4:~$ ps -efww | grep R | grep tomk
tomk 8366 13845 1 14:25 pts/000:00:01 /usr/lib/R/bin/exec/R
tomk 12960 27363 0 14:27 pts/300:00:00 grep --color=auto R
tomk@mr-0xb4:~$ grep Cpus /proc/8366/status
Cpus_allowed: 0001
Cpus_allowed_list: 0
As you can see, my R is super vanilla. I havent configured hardly anything.
Im just loading a few plain packages.
Thanks,
Tom
On Aug 6, 2014, at 2:20 PM, Nathaniel Smith wrote:
> On Wed, Aug 6, 2014 at 6:01 AM, Tom Kraljevic wrote:
>>
>> Hi,
>>
>>
>> (Using R 3.1.1 on Ubuntu 12.04.4 LTS)
>>
>>
>> What is the recommended way for R to fork a (non-R) process that is not CPU
>> limited?
>> Currently I am using R's system2() call, and this is inheriting the
>> environment of the R process.
>>
>>
>> I notice that (at least on Linux) when I am poking around /proc that the R
>> process itself is setting up cpu limitations for itself (max 1 core).
>>
>>
>> Using strace, I see the following:
>>
>> (strace output)
>> out.20612:sched_setaffinity(0, 128, {100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
>> 0, 0, 0, 0}) = 0
>>
>>
>> And proc shows:
>>
>> (cat /proc/nnn/status)
>> Cpus_allowed: 0001
>> Cpus_allowed_list: 0
>>
>>
>> See that the Cpus_allowed bitmask is a single core. Normally it's fff...f.
>
> When I run R I see:
>
> Cpus_allowed:ff
> Cpus_allowed_list:0-7
>
> It's possible (likely?) that the culprit here isn't R but rather some
> other library that R is loading. Are you using OpenBLAS? By default
> OpenBLAS will set an obnoxious cpu mask, unless you override this
> using some obscure build system settings. (There might be a runtime
> option for disabling it too, I don't remember. Note also that this is
> just one of several obnoxious things OpenBLAS does unless you override
> a bunch of obscure build system defaults -- building OpenBLAS
> correctly is highly non-trivial.)
>
> -n
>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R process (and forked children via system2) are limited to 1 core?
On Wed, Aug 6, 2014 at 10:31 PM, Tom Kraljevic wrote: > > Hi Nathaniel, > > > > Thanks for the suggestion. > > I’m actually not really using R to do any real work. > I’m starting the R H2O package (a Java machine learning package) and > forwarding all the work to H2O. > > > > This is the list of packages I have in R: > > >> search() > [1] ".GlobalEnv""package:h2o" "package:tools" > [4] "package:statmod" "package:rjson" "package:RCurl" > [7] "package:bitops""package:stats" "package:graphics" > [10] "package:grDevices" "package:utils" "package:datasets" > [13] "package:methods" "Autoloads" "package:base" > > > And here is the /proc info > > tomk@mr-0xb4:~$ ps -efww | grep R | grep tomk > tomk 8366 13845 1 14:25 pts/000:00:01 /usr/lib/R/bin/exec/R > tomk 12960 27363 0 14:27 pts/300:00:00 grep --color=auto R > tomk@mr-0xb4:~$ grep Cpus /proc/8366/status > Cpus_allowed: 0001 > Cpus_allowed_list: 0 > > > > As you can see, my R is super vanilla. I haven’t configured hardly > anything. I’m just loading a few plain packages. My suggestion is just a guess, really, but: R always uses (and thus loads by default) some underlying C library to implement its core linear algebra routines. OpenBLAS is one of the libraries that it might possibly be using, depending on how your R was set up. Anyway, it looks like the quick way to check for this particular possible culprit is to run env OPENBLAS_MAIN_FREE=1 R and see if that helps. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R process (and forked children via system2) are limited to 1 core?
Hi,
(Using R 3.1.1 on Ubuntu 12.04.4 LTS)
What is the recommended way for R to fork a (non-R) process that is not CPU
limited?
Currently I am using R’s system2() call, and this is inheriting the environment
of the R process.
I notice that (at least on Linux) when I am poking around /proc that the R
process itself is setting up cpu limitations for itself (max 1 core).
Using strace, I see the following:
(strace output)
out.20612:sched_setaffinity(0, 128, {100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0}) = 0
And proc shows:
(cat /proc/nnn/status)
Cpus_allowed: 0001
Cpus_allowed_list: 0
See that the Cpus_allowed bitmask is a single core. Normally it’s fff…f.
I want my child process (java in this case) not to share this limitation. What
is the recommended way of doing this from R?
Any ideas or suggestions appreciated!
Thanks,
Tom
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R process (and forked children via system2) are limited to 1 core?
Nathaniel, that did the trick. Thanks so much for your help. Tom $ export OPENBLAS_MAIN_FREE=1 $ R $ cat /proc/1538/status | grep Cpus_allowed Cpus_allowed: Cpus_allowed_list: 0-31 On Aug 6, 2014, at 2:41 PM, Nathaniel Smith wrote: > On Wed, Aug 6, 2014 at 10:31 PM, Tom Kraljevic wrote: >> >> Hi Nathaniel, >> >> >> >> Thanks for the suggestion. >> >> Im actually not really using R to do any real work. >> Im starting the R H2O package (a Java machine learning package) and >> forwarding all the work to H2O. >> >> >> >> This is the list of packages I have in R: >> >> >>> search() >> [1] ".GlobalEnv""package:h2o" "package:tools" >> [4] "package:statmod" "package:rjson" "package:RCurl" >> [7] "package:bitops""package:stats" "package:graphics" >> [10] "package:grDevices" "package:utils" "package:datasets" >> [13] "package:methods" "Autoloads" "package:base" >> >> >> And here is the /proc info >> >> tomk@mr-0xb4:~$ ps -efww | grep R | grep tomk >> tomk 8366 13845 1 14:25 pts/000:00:01 /usr/lib/R/bin/exec/R >> tomk 12960 27363 0 14:27 pts/300:00:00 grep --color=auto R >> tomk@mr-0xb4:~$ grep Cpus /proc/8366/status >> Cpus_allowed: 0001 >> Cpus_allowed_list: 0 >> >> >> >> As you can see, my R is super vanilla. I havent configured hardly >> anything. Im just loading a few plain packages. > > My suggestion is just a guess, really, but: R always uses (and thus > loads by default) some underlying C library to implement its core > linear algebra routines. OpenBLAS is one of the libraries that it > might possibly be using, depending on how your R was set up. > > Anyway, it looks like the quick way to check for this particular > possible culprit is to run > > env OPENBLAS_MAIN_FREE=1 R > > and see if that helps. > > -n > > -- > Nathaniel J. Smith > Postdoctoral researcher - Informatics - University of Edinburgh > http://vorpus.org [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] UTC time zone on Windows
Hi I'm having trouble running R CMD build and check with UTC time zone setting in Windows Server 2012. I can't seem to get rid of the following warning: unable to identify current timezone 'C': please set environment variable 'TZ' However, setting TZ to either "Europe/London" or "GMT Standard Time" didn't help. It seems to me that the warning originates in registryTZ.c (https://github.com/wch/r-source/blob/776708efe6003e36f02587ad47b2e19e2f69/src/extra/tzone/registryTZ.c#L363). I have therefore looked at HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation, to learn that TimeZoneKeyName is set to "UTC". This time zone is not defined in TZtable, but is present in this machine's HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones. (Also, the text of the warning permits the possibility that only the first character of the time zone is used for the warning message -- in the code, a const wchar_t* is used for a %s placeholder.) Below is a link to the log of such a failing run. The first 124 lines are registry dumps, output of R CMD * is near the end of the log at lines 212 and 224. https://ci.appveyor.com/project/krlmlr/r-appveyor/build/1.0.36 This happens with R 3.1.1 and R-devel r66309. Is there a workaround I have missed, short of updating TZtable? How can I help updating TZtable? Thanks! Cheers Kirill __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
