Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-20 Thread Gabriel Becker
Mario, Abby, et al. Note that there is no fully safe way of unloading packages which register methods (as answered by Luke Tierney here: https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16644 ) which makes the single R session running arbitrary different scripts thing pretty iffy over the long

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-20 Thread Abby Spurdle
Thank you Serguei and Gabor. Great suggestions. > If your R scripts contain "stop()" or "q('yes')" or any other error, it > will end the Rscript process. Kind of watch-dog can be set for automatic > relaunching if needed. It should be possible to change the error handling behavior. >From within R

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-20 Thread Serguei Sokol
Le 20/07/2020 à 10:15, Abby Spurdle a écrit : It's possible to run R (or a c parent process) as a background process via a named pipe, and then write script files to the named pipe. However, the details depend on what shell you use. The last time I tried (which was a long time ago), I created a

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-20 Thread Gábor Csárdi
On Mon, Jul 20, 2020 at 9:15 AM Abby Spurdle wrote: > > It's possible to run R (or a c parent process) as a background process > via a named pipe, and then write script files to the named pipe. > However, the details depend on what shell you use. I would use screen or tmux for this, if this is an

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-20 Thread Abby Spurdle
It's possible to run R (or a c parent process) as a background process via a named pipe, and then write script files to the named pipe. However, the details depend on what shell you use. The last time I tried (which was a long time ago), I created a small c program to run R, read from the named pi

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Tobias Verbeke
- Original Message - > From: "Dirk Eddelbuettel" > To: "Mario Annau" > Cc: "r-devel@r-project.org" > Sent: Sunday, July 19, 2020 10:09:24 PM > Subject: Re: [Rd] Speed-up/Cache loadNamespace() > On 19 July 2020 at 20:47, Mario Annau wrot

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Dirk Eddelbuettel
On 19 July 2020 at 20:47, Mario Annau wrote: | Am So., 19. Juli 2020 um 20:11 Uhr schrieb Hugh Parsonage < | hugh.parson...@gmail.com>: | > 3. Keep an R session running in perpetuity and source the scripts within | > that everlasting session | However, 3. sounds interesting - how would this work

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Simon Urbanek
Mario, On unix if you use Rseve you can pre-load all packages in the server (via eval config directive or by running Rserve::run.Rserve() from a session that has everything loaded) and all client connections will have the packages already loaded and available* immediately. You could replace Rsc

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Mario Annau
Thanks for the quick responses. As you both suggested storing the packages to local drive is feasible but comes with a size restriction I wanted to avoid. I'll keep this in mind as plan B. @Hugh: 2. would impose even greater slowdowns and 4. is just not feasible. However, 3. sounds interesting - ho

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Hugh Parsonage
My advice would be to avoid the network in one of the following ways 1. Store installed packages on your local drive 2. Copy the installed packages to a tempdir on your local drive each time the script is executed 3. Keep an R session running in perpetuity and source the scripts within that everla

Re: [Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Duncan Murdoch
On 19/07/2020 11:50 a.m., Mario Annau wrote: Dear all, in our current setting we have our packages stored on a (rather slow) network drive and need to invoke short R scripts (using RScript) in a timely manner. Most of the script's runtime is spent with package loading using library() (or loadNam

[Rd] Speed-up/Cache loadNamespace()

2020-07-19 Thread Mario Annau
Dear all, in our current setting we have our packages stored on a (rather slow) network drive and need to invoke short R scripts (using RScript) in a timely manner. Most of the script's runtime is spent with package loading using library() (or loadNamespace to be precise). Is there a way to cache