Re: [Rd] Excessive network load
You haven't even told us your OS! The simple solution is to use a local copy of R. It is probably the case that few of those 1500 packages are used, and certainly that few are used in each R session: you may also want to install locally any that are heavily used. We found it advantageous to do this for Windows users of R (where the remote discs are mounted by SMB) quite a while ago, and more recently moved to local installation of R on Linux machines. The issue was not network load but latency for interactive users: although R is heavily used here it is not a major component of our network load and we are rather protecting R users against other applications that are much more demanding. On Mon, 23 Oct 2006, Claudio Lottaz wrote: > Dear all, > > I wonder, if anybody experiences similar problems and if there are any > simple solutions to be suggested. We observe that R causes a lot of > network traffic and thus slows down the performance of the whole > network. When tracing the network traffic on the machine which serves > the R installation via NFS, we see thousands of requests at > initialization of R processes and regular calls, probably to shared > libraries. Is there a way to compile or run R such that it causes less > load on the network? > > Here is some information on our installation: > - We use a single installation of R (version 2.3.1) loaded over NFS > - there are approximately 1500 packages installed using ~8GB of disk > - We use R on a queuing system running up to 50 processes in parallel > > The load on the machine which serves the R installation frequently rises up > to 5 or so although it is a dedicated machine. > Any hints towards measures against network load are highly appreciated. > > Thanks, > Claudio > > > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] ANN/RFC: package providing TAB completion for readline-based R
Deepayan Sarkar wrote: > Announcement: > > (For those every-once-in-a-while occasions when you run R from a > terminal instead of Emacs, and then wish something would happen when > you hit TAB...) > > Last week, I started looking at the GNU Readline documentation to see > if I could figure out how to use it for command completion within R. > It turned out to be easier than I had expected, and I now have a beta > version of the 'rcompletion' package available at > > http://www.stat.wisc.edu/~deepayan/R/rcompletion_0.0-6.tar.gz > >>From the help page: > > Description: > > This package provides pseudo-intelligent TAB completion for a > readline enabled instance of R when it is run from a terminal (or > more specifically, an interface which uses readline to accept user > input). It has no effect on the various GUI interfaces to R, > including ESS and the standard Windows interface. > > For more details, install and load the package, then type > > package?rcompletion > > at the R prompt. Testing and feedback would be appreciated. Feature > requests and patches are also welcome, of course. > > > Questions: > > There are a couple of things about which I would like some advice: > > (1) The package currently contains a very rudimentary configure script > which stops installation when readline is not found. I'm not sure if > this is portable enough. .../src/unix/sys-std.c has more sophisticated > conditional directives, but I don't know enough about this sort of > thing to interpret those. Any thoughts or suggestions on this would be > appreciated. > > (2) What's the recommended procedure to distribute this, given that > (a) it's not relevant on Windows (i.e. won't do anything even if it > could be installed) and (b) may or may not be relevant on Macs, some > of which apparently have a fake readline that doesn't support > completion? Distribution can happen regularly as a CRAN package. I will put it on my exclude list in order not to try building Windows binaries in that case. Uwe > -Deepayan > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] ANN/RFC: package providing TAB completion for readline-based R
On Tue, 24 Oct 2006, Uwe Ligges wrote: > > > Deepayan Sarkar wrote: >> Announcement: >> >> (For those every-once-in-a-while occasions when you run R from a >> terminal instead of Emacs, and then wish something would happen when >> you hit TAB...) >> >> Last week, I started looking at the GNU Readline documentation to see >> if I could figure out how to use it for command completion within R. >> It turned out to be easier than I had expected, and I now have a beta >> version of the 'rcompletion' package available at >> >> http://www.stat.wisc.edu/~deepayan/R/rcompletion_0.0-6.tar.gz >> >>> From the help page: >> >> Description: >> >> This package provides pseudo-intelligent TAB completion for a >> readline enabled instance of R when it is run from a terminal (or >> more specifically, an interface which uses readline to accept user >> input). It has no effect on the various GUI interfaces to R, >> including ESS and the standard Windows interface. >> >> For more details, install and load the package, then type >> >> package?rcompletion >> >> at the R prompt. Testing and feedback would be appreciated. Feature >> requests and patches are also welcome, of course. >> >> >> Questions: >> >> There are a couple of things about which I would like some advice: >> >> (1) The package currently contains a very rudimentary configure script >> which stops installation when readline is not found. I'm not sure if >> this is portable enough. .../src/unix/sys-std.c has more sophisticated >> conditional directives, but I don't know enough about this sort of >> thing to interpret those. Any thoughts or suggestions on this would be >> appreciated. I don't think that is safe. You can have R installed with readline support even if there are neither readline headers nor library on the current machine (use a static library), and R can be run without readline (via --no-readline) even if it is installed. However, your configure script fails on my machine (x86_64 FC5). It seems that you need both to get LDFLAGS and LD_LIBRARY_PATH from R and use them, and also use a version of autoconf that knows about lib64 (since the readline DSO is in /usr/lib64). Since I don't get any further I am not sure what really happens, but it seems you don't link your DLL against -lreadline, and I think you need to to allow for static libreadline.a (the relevant code seems to be a separate .o and so is likely not linked into R in that case). I guess that you want to know that R was installed with readline support at install time, and is running with the readline console? It is not immediately obvious to me that either is currently known to a package writer/user. >> (2) What's the recommended procedure to distribute this, given that >> (a) it's not relevant on Windows (i.e. won't do anything even if it >> could be installed) and (b) may or may not be relevant on Macs, some >> of which apparently have a fake readline that doesn't support >> completion? > > > Distribution can happen regularly as a CRAN package. I will put it on my > exclude list in order not to try building Windows binaries in that case. I think there should also be a configure.win which explains that the package is not usable on Windows, in case someone tries to install from the sources. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Variables ordering problem in mle() (PR#9313)
Full_Name: Sébastien Villemot Version: 2.4.0 OS: Debian testing Submission from: (NULL) (62.212.121.128) Hi, In the mle() function of the stats4 package, there is a bug in the ordering of the variables given in the 'start' argument. By just changing the order of the variables listed in the 'start' list (the initialization values), it is possible to obtain different estimation results (actually different local minima). The bug is easy to reproduce, with the following example: > library(stats4) > g <- function(x, y) -cos(x) + abs(y) > mle(g, start = list(x = 0, y = 7)) Call: mle(minuslogl = g, start = list(x = 0, y = 7)) Coefficients: xy 0.00e+00 1.409463e-18 > mle(g, start = list(y = 7, x = 0)) Call: mle(minuslogl = g, start = list(y = 7, x = 0)) Coefficients: xy 6.283185 0.00 The problem comes from the reordering of the 'start' list at the following line of the body of 'mle': start <- start[order(oo)] which is not accompanied by a reordering of the 'nm' list used in the 'f' local function at: names(l) <- nm Greets, S.V. > sessionInfo() R version 2.4.0 (2006-10-03) i486-pc-linux-gnu locale: LC_CTYPE=fr_FR.UTF-8;LC_NUMERIC=C;LC_TIME=fr_FR.UTF-8;LC_COLLATE=fr_FR.UTF-8;LC_MONETARY=fr_FR.UTF-8;LC_MESSAGES=fr_FR.UTF-8;LC_PAPER=fr_FR.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=fr_FR.UTF-8;LC_IDENTIFICATION=C attached base packages: [1] "stats4""methods" "stats" "graphics" "grDevices" "utils" [7] "datasets" "base" __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Statusbar help sometimes displays wrong function (PR#9314)
Full_Name: Hendrik Fuß Version: 2.4.0 OS: Mac OS 10.4.8 Submission from: (NULL) (201.43.91.248) Steps to reproduce the bug: Define the function my.mean = function(x,y) (x+y)/2 then type "my.m" in the interactive shell and press TAB. "my.mean(" appears at the input prompt, but the status bar displays synopsis for mean() rather than my.mean(). __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] as.missing
(I'm not sure if this is a request for a feature, or another instance where a feature has eluded me for many years.) Often I have a function which calls other functions, and may often use the default arguments to those functions, but needs the capability to pass along non-default choices. I usually do this with some variation on foo <- function(x, foo2Args=NULL or a list(foo2defaults), foo3Args=NULL or a list(foo3defaults)) and then have logic to check for NULL, or use the list in combination with do.call. It is also possible to do this with ..., but it always seems a bit dangerous passing all the unnamed arguments along to all the functions being called, especially when I always seem to be calling functions that have similar arguments (maxit, eps, start, frequency, etc). It is a situation I have learned to live with, but one of my co-maintainers just pointed out to me that there should be a good way to do this in R. Perhaps there is something else I have missed all these years? Is there a way to do this cleanly? It would be nice to have something like foo <- function(x, foo2Args=as.missing(), foo3Args=as.missing()) then the call to foo2 and foo3 could specify foo2Args and foo3Args, but these would get treated as if they were missing, unless they are given other values. Paul Gilbert La version française suit le texte anglais. This email may contain privileged and/or confidential inform...{{dropped}} __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] as.missing
On 10/24/2006 12:58 PM, Paul Gilbert wrote: > (I'm not sure if this is a request for a feature, or another instance > where a feature has eluded me for many years.) > > Often I have a function which calls other functions, and may often use > the default arguments to those functions, but needs the capability to > pass along non-default choices. I usually do this with some variation on > > foo <- function(x, foo2Args=NULL or a list(foo2defaults), > foo3Args=NULL or a list(foo3defaults)) > > and then have logic to check for NULL, or use the list in combination > with do.call. It is also possible to do this with ..., but it always > seems a bit dangerous passing all the unnamed arguments along to all the > functions being called, especially when I always seem to be calling > functions that have similar arguments (maxit, eps, start, frequency, etc). > > It is a situation I have learned to live with, but one of my > co-maintainers just pointed out to me that there should be a good way to > do this in R. Perhaps there is something else I have missed all these > years? Is there a way to do this cleanly? It would be nice to have > something like > > foo <- function(x, foo2Args=as.missing(), foo3Args=as.missing()) > > then the call to foo2 and foo3 could specify foo2Args and foo3Args, but > these would get treated as if they were missing, unless they are given > other values. I was going to say I couldn't see the difference between this and just declaring foo <- function(x, foo2Args, foo3Args) with no defaults. However, this little demo illustrates the point, I think: > g <- function(gnodef, gdef=1) { +if (missing(gnodef)) cat('gnodef is missing\n') +if (missing(gdef)) cat('gdef is missing\n') +cat('gdef is ',gdef,'\n') + } > > f <- function(fnodef, fdef) { +g(fnodef, fdef) + } > > g() gnodef is missing gdef is missing gdef is 1 > f() gnodef is missing gdef is missing Error in cat("gdef is ", gdef, "\n") : argument "fdef" is missing, with no default What would be nice to be able to do is to have a simple way for f() to act just like g() does. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] as.missing
On Tue, 24 Oct 2006, Duncan Murdoch wrote: > On 10/24/2006 12:58 PM, Paul Gilbert wrote: >> (I'm not sure if this is a request for a feature, or another instance >> where a feature has eluded me for many years.) >> >> Often I have a function which calls other functions, and may often use >> the default arguments to those functions, but needs the capability to >> pass along non-default choices. I usually do this with some variation on >> >> foo <- function(x, foo2Args=NULL or a list(foo2defaults), >> foo3Args=NULL or a list(foo3defaults)) >> >> and then have logic to check for NULL, or use the list in combination >> with do.call. It is also possible to do this with ..., but it always >> seems a bit dangerous passing all the unnamed arguments along to all the >> functions being called, especially when I always seem to be calling >> functions that have similar arguments (maxit, eps, start, frequency, etc). >> >> It is a situation I have learned to live with, but one of my >> co-maintainers just pointed out to me that there should be a good way to >> do this in R. Perhaps there is something else I have missed all these >> years? Is there a way to do this cleanly? It would be nice to have >> something like >> >> foo <- function(x, foo2Args=as.missing(), foo3Args=as.missing()) >> >> then the call to foo2 and foo3 could specify foo2Args and foo3Args, but >> these would get treated as if they were missing, unless they are given >> other values. > > I was going to say I couldn't see the difference between this and just > declaring > > foo <- function(x, foo2Args, foo3Args) > > with no defaults. However, this little demo illustrates the point, I think: > > > g <- function(gnodef, gdef=1) { > +if (missing(gnodef)) cat('gnodef is missing\n') > +if (missing(gdef)) cat('gdef is missing\n') > +cat('gdef is ',gdef,'\n') > + } > > > > f <- function(fnodef, fdef) { > +g(fnodef, fdef) > + } > > > > g() > gnodef is missing > gdef is missing > gdef is 1 > > f() > gnodef is missing > gdef is missing > Error in cat("gdef is ", gdef, "\n") : argument "fdef" is missing, with > no default > > > What would be nice to be able to do is to have a simple way for f() to > act just like g() does. Is this what you want? > f <- function(fnodef, fdef=NULL) { + g()} > f() gnodef is missing gdef is missing gdef is 1 > > > Duncan Murdoch > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > Charles C. Berry(858) 534-2098 Dept of Family/Preventive Medicine E mailto:[EMAIL PROTECTED] UC San Diego http://biostat.ucsd.edu/~cberry/ La Jolla, San Diego 92093-0717 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] as.missing
"Charles C. Berry" <[EMAIL PROTECTED]> writes: > On Tue, 24 Oct 2006, Duncan Murdoch wrote: > > with no defaults. However, this little demo illustrates the point, I think: > > > > > g <- function(gnodef, gdef=1) { > > +if (missing(gnodef)) cat('gnodef is missing\n') > > +if (missing(gdef)) cat('gdef is missing\n') > > +cat('gdef is ',gdef,'\n') > > + } > > > > > > f <- function(fnodef, fdef) { > > +g(fnodef, fdef) > > + } > > > > > > g() > > gnodef is missing > > gdef is missing > > gdef is 1 > > > f() > > gnodef is missing > > gdef is missing > > Error in cat("gdef is ", gdef, "\n") : argument "fdef" is missing, with > > no default > > > > > > What would be nice to be able to do is to have a simple way for f() to > > act just like g() does. > > > Is this what you want? > > > f <- function(fnodef, fdef=NULL) { > + g()} > > f() > gnodef is missing > gdef is missing > gdef is 1 I think not. More like f <- function(fnodef, fdef) { if(missing(fdef)) g(fnodef) else g(fnodef, fdef) } (the generalization of which is obviously a pain...) -- O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Cook's Distance in GLM (PR#9316)
Hi Community, I'm trying to reconcile Cook's Distances computed in glm. The following snippet of code shows that the Cook's Distances contours on the plot of Residuals v Leverage do not seem to be the same as the values produced by cooks.distance() or in the Cook's Distance against observation number plot. counts <- c(18,17,15,20,10,20,25,13,12) outcome <- gl(3,1,9) treatment <- gl(3,3) d.AD <- data.frame(treatment, outcome, counts) glm.D93 <- glm(counts ~ outcome + treatment, family=poisson()) opar <- par(mfrow=c(2,1)) plot(glm.D93, which=c(4,5)) par(opar) cooks.distance(glm.D93) The difference is reasonably moderate in this case. My suspicions were aroused by a case in which the plot showed five or size points greater than 1, none of which could be identified in the output of the function. > version _ platform i386-unknown-freebsd6.1 arch i386 os freebsd6.1 system i386, freebsd6.1 status Patched major 2 minor 4.0 year 2006 month 10 day03 svn rev39576 language R version.string R version 2.4.0 Patched (2006-10-03 r39576) Cheers Andrew -- Andrew Robinson Department of Mathematics and StatisticsTel: +61-3-8344-9763 University of Melbourne, VIC 3010 Australia Fax: +61-3-8344-4599 http://www.ms.unimelb.edu.au/~andrewpr http://blogs.mbs.edu/fishing-in-the-bay/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Cook's Distance in GLM (PR#9316)
[EMAIL PROTECTED] writes: > Hi Community, > > I'm trying to reconcile Cook's Distances computed in glm. The > following snippet of code shows that the Cook's Distances contours on > the plot of Residuals v Leverage do not seem to be the same as the > values produced by cooks.distance() or in the Cook's Distance against > observation number plot. > > counts <- c(18,17,15,20,10,20,25,13,12) > outcome <- gl(3,1,9) > treatment <- gl(3,3) > d.AD <- data.frame(treatment, outcome, counts) > glm.D93 <- glm(counts ~ outcome + treatment, family=poisson()) > > opar <- par(mfrow=c(2,1)) > plot(glm.D93, which=c(4,5)) > par(opar) > > cooks.distance(glm.D93) > > The difference is reasonably moderate in this case. My suspicions > were aroused by a case in which the plot showed five or size points > greater than 1, none of which could be identified in the output of the > function. Hmm, yes. A good guess is that the contour levels need to be modified by a dispersion factor. The plot is much more consistent with cooks.distance(glm.D93,dispersion=5.129/4 ) -- O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] stack imbalance in contour
Hi Peter Dalgaard wrote: > Patrick Burns <[EMAIL PROTECTED]> writes: > >> I'm not sure if this has much significance or not -- but >> it sounds rather ominous. It doesn't appear to be new >> as it happens with 2.0.0 in Linux (but the formatting of >> the warning messages has improved). >> >> > contour(array(3e10, c(10,10), list(1:10, 11:20))) >> Warning: stack imbalance in 'contour', 20 then 24 >> Warning: stack imbalance in '.Internal', 19 then 23 >> Warning: stack imbalance in '{', 17 then 21 >> Warning message: >> all z values are equal >> > sessionInfo() >> R version 2.4.0 RC (2006-09-27 r39543) >> i386-pc-mingw32 > > I see it on Linux too. It's a serious programming error in that > something has its PROTECT/UNPROTECT mechanism messed up. Fortunately, > it is protecting too much, which is not usually fatal. Also, it > appears to happen only in the all z equal case (the value 3e10 is > not magic - any nonzero z will do). We should get it tracked down, > but it's not worth breaking the code freeze for 2.4.0 over. Now fixed in r-devel and r-patched. Paul -- Dr Paul Murrell Department of Statistics The University of Auckland Private Bag 92019 Auckland New Zealand 64 9 3737599 x85392 [EMAIL PROTECTED] http://www.stat.auckland.ac.nz/~paul/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] ANN/RFC: package providing TAB completion for readline-based R
On 10/24/06, Prof Brian Ripley <[EMAIL PROTECTED]> wrote: > On Tue, 24 Oct 2006, Uwe Ligges wrote: [...] > >> Questions: > >> > >> There are a couple of things about which I would like some advice: > >> > >> (1) The package currently contains a very rudimentary configure script > >> which stops installation when readline is not found. I'm not sure if > >> this is portable enough. .../src/unix/sys-std.c has more sophisticated > >> conditional directives, but I don't know enough about this sort of > >> thing to interpret those. Any thoughts or suggestions on this would be > >> appreciated. > > I don't think that is safe. You can have R installed with readline > support even if there are neither readline headers nor library on the > current machine (use a static library), and R can be run without readline > (via --no-readline) even if it is installed. I don't think the last case is a problem; nothing useful will happen after loading the package in that case, but nothing bad will happen either. If there's no readline library, is there anything the configure script can reasonably do other than abort? If the library is there but the headers aren't, I can probably use headers included in the package. Is that a reasonable approach? I have no idea what to do with a static library. > However, your configure script fails on my machine (x86_64 FC5). Fails in what sense? Does it abort saying it didn't find readline? > It seems > that you need both to get LDFLAGS and LD_LIBRARY_PATH from R and use them, > and also use a version of autoconf that knows about lib64 (since the > readline DSO is in /usr/lib64). Since I don't get any further I am not > sure what really happens, but it seems you don't link your DLL against > -lreadline, and I think you need to to allow for static libreadline.a (the > relevant code seems to be a separate .o and so is likely not linked into R > in that case). I have modified configure.in to add a -lreadline to PKG_LIBS, and also to use header files from within the package if the system headers are not found. The new version is at http://www.stat.wisc.edu/~deepayan/R/rcompletion_0.0-7.tar.gz I'm working on a 64 bit system, so I'm hoping autoconf knows what to do there. I'm not sure how to get or use LDFLAGS and LD_LIBRARY_PATH from R. R CMD config LDFLAGS does give me something (but I'm not sure what to do with it), but there doesn't seem to be anything analogous to LD_LIBRARY_PATH. > I guess that you want to know that R was installed with readline support > at install time, and is running with the readline console? It is not > immediately obvious to me that either is currently known to a package > writer/user. I don't think that's as much of an issue. If R doesn't ever make a call to readline(), whether the rcompletion package is loaded or not won't make any difference. The user won't get the benefits, but that's a choice made by the user. For example, running R through ESS and then loading rcompletion doesn't have any effect at all. The one scenario that definitely needs to be avoided is where R CMD INSTALL appears to install the package successfully and then library(rcompletion) causes an error (either immediately on load or later). Unless anyone reports seeing that, I'll plan to upload this to CRAN in a few days. Cases where configure aborts even though it shouldn't will probably need some time (and help) to work out, since I'm completely new to autotools. > >> (2) What's the recommended procedure to distribute this, given that > >> (a) it's not relevant on Windows (i.e. won't do anything even if it > >> could be installed) and (b) may or may not be relevant on Macs, some > >> of which apparently have a fake readline that doesn't support > >> completion? > > > > > > Distribution can happen regularly as a CRAN package. I will put it on my > > exclude list in order not to try building Windows binaries in that case. > > I think there should also be a configure.win which explains that the > package is not usable on Windows, in case someone tries to install from > the sources. If configure.win doesn't exist, would it try to use configure? In that case, installation should just abort when readline isn't found, shouldn't it? Would having a configure.win serve any additional purpose? -Deepayan __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel