[Rd] robustness of install.packages/update.packages (was Re: bug in L-BFGS-B? (PR#8099))
G'day Brian, I am splitting my reply to your e-mail into two since there are two separate spinoffs. > "BDR" == Prof Brian Ripley <[EMAIL PROTECTED]> writes: BDR> Check your versions of MASS. The Windows one appears to be BDR> an outdated version, and does different things. Thanks, yes, this is the problem. On my linux box, I compile things from source and had the latest version of MASS (7.2-19) installed under R 2.1.1 and R 2.2.0. Since I hardly ever work under Windows, I rely for my Windows box on the precompiled versions provided from CRAN and do not compile R or packages from source. (Though I have it set up so that I can compile my own packages.) Actually, I only installed R 2.1.1 on that machine (which has an AMD processor) yesterday and it was using MASS version 7.2-16. Running `update.packages' today, updated MASS to version 7.2-18. And, lo and behold, there are no more warning messages, even more, running the example in `fitdistr' now returns: > set.seed(123) > x <- rgamma(100, shape = 5, rate = 0.1) > fitdistr(x, "gamma") shape rate 6.45947303 0.13593172 (0.89052006) (0.01948648) > ## now do this directly with more control. > fitdistr(x, dgamma, list(shape = 1, rate = 0.1), lower = 0.01) shape rate 6.48714685 0.13651706 (0.89442924) (0.01956916) So it was perhaps lucky that I did not run update.packages yesterday, since I would have expected that (like PR#1717) my bug report would be filed under not-reproducible. :-) But this made me curious, and I fired my laptop (which has an Intel chip) up under Windows (usually it runs under Linux) and installed R 2.1.1 on it. On that Windows machine, I get the same answers as those reported yesterday under both MASS 7.2-16 and 7.2-18. Thus, I am a bit baffled. Well, I just went back to the AMD box and re-run the code, now it gives me again the "nonsense" answer in the second case. There is definitely something wrong in the L-BFGS-B routine, which will be hard to find. Regarding the changed subject line: After installing R 2.1.1 on my laptop, I executed an R source file which, essentially, only contains an install.packages command with the list of contributed packages that I like to have installed. One of this packages is DAAG and it seems that overnight the file DAAG_0.58.zip disappeared from http://mirror.aarnet.edu.au/pub/CRAN/bin/windows/contrib/2.1/ so the install.packages command terminated with an error message that the source file could not be downloaded. Would it be possible to robustify the install.packages command such that it continues with installing all the other packages requested instead of terminating? After redirecting R 2.1.1 on my laptop to use http://cran.au.r-project.org/ for the CRAN repository, the install.packages() command ran without problems. I issued the command `library(MASS)' and tried out the example from fitdistr on that machine (same strange result for second command and warning messages were issued). So I said update.packages() and that command failed when it wanted to update the MASS package. So I detach()'ed MASS and re-ran update.packages() and again it failed. So I exited R 2.1.1 and restarted it again (probably I should have unloaded the namespace of MASS??) and then the update.packages command worked. However, update.packages() wanted to update quite a few packages besides MASS (the other packages in the VR bundle, nlme, lattice &c). Once it failed on MASS, it terminated with an error and did not update any of the other packages. Would it be possible to robustify update.packages behaviour such that it would continue in such situations with updating the remaining packages? Cheers, Berwin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Coding standards (was Re: bug in L-BFGS-B? (PR#8099))
G'day Brian,
> "BDR" == Prof Brian Ripley <[EMAIL PROTECTED]> writes:
BDR> As for the problem, yes it probably is a bug in L-BFGS-B.
BDR> Fancy debugging the code?
I was afraid that somebody would ask this. ;-)
I looked a bit at the code and it seems to be non-trivial. Moreover,
it seems to be code translated from FORTRAN to C via f2c and I am not
a fan of such kind of code. I know that "Writing R Extensions" lists
in Appendix B ("R coding standards") f2c as a tool that `can "safely
be assumed" for R extensions'. However, f2c, despite its `-A' option,
does not produce ANSI C compliant code but rather C code that provokes
undefined behaviour.
The problem is, that the code produced by f2c is decrementing pointers
to simulate 1-based vectors and the C FAQ is pretty unambigious about
the fact that this provokes undefined behaviour, see
http://www.eskimo.com/~scs/C-faq/q6.17.html
As far as I understand, this translated code mostly stems from the
time when some platforms did not have ready access to a fortran
compiler and, hence, f2c was used (extensively?). But now, with g77
this does not seem to be an issue anymore. So I wonder whether there
are any plans of returning to the original fortran code? Or are there
plans to clean up these f2c'd code snippet to make them ANSI C
compliant?
I noticed such f2c'd code already in the splines.c file when I studied
how splinefun was implemented (which lead to bug report #8030). In
that case, I am fairly familiar with the algorithms used in splines.c
since I programmed them myself on other occassions and I probably
could rewrite the algorithms in proper ANSI C (it would still take
some time). But it would be nice to know what the official stance of
the R Core Team is.
When I spoke with one member of the R Core Team about this on a
conference in 2004, the answer was, IIRC, "yes, we know that this code
invokes undefined behaviour, but there are bigger problems to fix
first and this pointer manipulation seems to work on all platforms
these days." Another member of the R Core Team whom I recently asked:
I guess all platforms on which R is running at the moment do not
have a problem with this trick, but are there any plans to
change such kind of code to "valid" C? Would patches that do
that be accepted?
answered:
Hmm, I think we'd tend to disagree here. But in any case that
would be a wide issue. Can you address this question to R-core,
please? (or I forward?)
Cheers,
Berwin
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] xerbla called from BLAS routine (PR#8100)
Full_Name: Göran Broström
Version: R-2.1.1, 2.2.0
OS: Debian unstable
Submission from: (NULL) (213.65.9.59)
Some BLAS routines call xerbla for error messages, which results in a message
like
"LAPACK routine DGER gave error code -9".
Suggested solution: In
void F77_NAME(xerbla)(char *srname, int *info)
{
/* srname is not null-terminated. It should be 6 characters. */
char buf[7];
strncpy(buf, srname, 6);
buf[6] = '\0';
error(_("LAPACK routine '%6s' gave error code %d"), buf, -(*info));
}
change 'LAPACK' to 'The' (or 'The BLAS/LAPACK').
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Coding standards (was Re: bug in L-BFGS-B? (PR#8099))
Berwin A Turlach <[EMAIL PROTECTED]> writes: > The problem is, that the code produced by f2c is decrementing pointers > to simulate 1-based vectors and the C FAQ is pretty unambigious about > the fact that this provokes undefined behaviour, see > http://www.eskimo.com/~scs/C-faq/q6.17.html > > As far as I understand, this translated code mostly stems from the > time when some platforms did not have ready access to a fortran > compiler and, hence, f2c was used (extensively?). But now, with g77 > this does not seem to be an issue anymore. So I wonder whether there > are any plans of returning to the original fortran code? Or are there > plans to clean up these f2c'd code snippet to make them ANSI C > compliant? There's more to it than that. F2c has also been used in places to make Fortran code reentrant, which it is generally not. I forget whether we were ever bitten by nested calls to optim(), but we certainly was by integrate(). There is of course no argument against the fact that the pointer-decrement trick goes against the C specs, but I do wonder whether it is *actually* biting anyone these days. The remarks in the C FAQ seem to mainly refer to segmented architectures like the original IBM PC. With modern flat address spaces, it would seem quite unlikely to run into trouble. -- O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] .Call and Segmentation Fault
> It cannot be .Call: your .Call passed no parameters so there was nothing > to be uninitialized. I did ask you why you were doing that. This .Call without parameters is just for a test. I am constructing an interface to this C/Fortran optimization package, and because this segmentation fault problem, I decided to modify the interface for a simple test: call a C function that initialize the set of variables wich describe the problem to be solved (wich supose to be done in R) and call the optimization package. But my surprise was that the program when called from R gives me the same segmentation fault error. But when I compile the program as a regular linux binary and run it from shell I don't get any problem. That's why I thought it could be a problem with .Call. > .Call is very heavily tested in lots of R applications, so the prior > probability of innocence must be very high. Thanks professor, I will check the C program again. Ricardo > > -- > Brian D. Ripley, [EMAIL PROTECTED] > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > University of Oxford, Tel: +44 1865 272861 (self) > 1 South Parks Road, +44 1865 272866 (PA) > Oxford OX1 3TG, UKFax: +44 1865 272595 > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] robustness of install.packages/update.packages (was Re: bug in L-BFGS-B? (PR#8099))
We've never encountered this lying mirror problem. Perhaps you (or another user of the unreliable mirror) could contribute suitable fixes. On Sun, 28 Aug 2005, Berwin A Turlach wrote: > G'day Brian, > > I am splitting my reply to your e-mail into two since there are two > separate spinoffs. > >> "BDR" == Prof Brian Ripley <[EMAIL PROTECTED]> writes: > >BDR> Check your versions of MASS. The Windows one appears to be >BDR> an outdated version, and does different things. > Thanks, yes, this is the problem. On my linux box, I compile things > from source and had the latest version of MASS (7.2-19) installed > under R 2.1.1 and R 2.2.0. > > Since I hardly ever work under Windows, I rely for my Windows box on > the precompiled versions provided from CRAN and do not compile R or > packages from source. (Though I have it set up so that I can compile > my own packages.) Actually, I only installed R 2.1.1 on that machine > (which has an AMD processor) yesterday and it was using MASS version > 7.2-16. Running `update.packages' today, updated MASS to version > 7.2-18. And, lo and behold, there are no more warning messages, even > more, running the example in `fitdistr' now returns: > >> set.seed(123) >> x <- rgamma(100, shape = 5, rate = 0.1) >> fitdistr(x, "gamma") > shape rate > 6.45947303 0.13593172 > (0.89052006) (0.01948648) >> ## now do this directly with more control. >> fitdistr(x, dgamma, list(shape = 1, rate = 0.1), lower = 0.01) > shape rate > 6.48714685 0.13651706 > (0.89442924) (0.01956916) > > So it was perhaps lucky that I did not run update.packages yesterday, > since I would have expected that (like PR#1717) my bug report would be > filed under not-reproducible. :-) > > But this made me curious, and I fired my laptop (which has an Intel > chip) up under Windows (usually it runs under Linux) and installed R > 2.1.1 on it. On that Windows machine, I get the same answers as those > reported yesterday under both MASS 7.2-16 and 7.2-18. Thus, I am a > bit baffled. Well, I just went back to the AMD box and re-run the > code, now it gives me again the "nonsense" answer in the second case. > There is definitely something wrong in the L-BFGS-B routine, which > will be hard to find. > > Regarding the changed subject line: > > After installing R 2.1.1 on my laptop, I executed an R source file > which, essentially, only contains an install.packages command with the > list of contributed packages that I like to have installed. One of > this packages is DAAG and it seems that overnight the file > DAAG_0.58.zip disappeared from >http://mirror.aarnet.edu.au/pub/CRAN/bin/windows/contrib/2.1/ > so the install.packages command terminated with an error message that > the source file could not be downloaded. > > Would it be possible to robustify the install.packages command such > that it continues with installing all the other packages requested > instead of terminating? > > After redirecting R 2.1.1 on my laptop to use >http://cran.au.r-project.org/ > for the CRAN repository, the install.packages() command ran without > problems. I issued the command `library(MASS)' and tried out the > example from fitdistr on that machine (same strange result for second > command and warning messages were issued). So I said > update.packages() and that command failed when it wanted to update > the MASS package. So I detach()'ed MASS and re-ran update.packages() > and again it failed. So I exited R 2.1.1 and restarted it again > (probably I should have unloaded the namespace of MASS??) and then the > update.packages command worked. Yes, and that *is* in the rw-FAQ. > However, update.packages() wanted to update quite a few packages > besides MASS (the other packages in the VR bundle, nlme, lattice &c). > Once it failed on MASS, it terminated with an error and did not update > any of the other packages. Would it be possible to robustify > update.packages behaviour such that it would continue in such > situations with updating the remaining packages? Not a good idea. Better to follow the FAQ. At that point the dependencies have been worked out and will not be re-computed if a package installation fails. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Additional Books on R for Web Site Listing
Hi all,
I noted that the books below are not included on the R web site
Documentation section under "Books". I thought that I would provide
these and have created some bib entries consistent with the style used
on the site. The abstract sections were copied from the publisher sites.
I hope that these are acceptable to both the book list maintainer and
the respective authors. Feel free to modify as may be appropriate.
It is great to see the increasing number of new books on R!
A special note to Paul Murrell on a great addition to the R library! I
just got my copy this past week from Amazon.com. Well Done!
Best regards,
Marc Schwartz
@BOOK{R:Crawley:2005,
AUTHOR = {Michael J. Crawley},
TITLE = {Statistics: An Introduction using R},
PUBLISHER = {Wiley},
YEAR = 2005,
NOTE = {ISBN 0-470-02297-3},
PUBLISHERURL =
{http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470022973.html},
URL = {http://www.bio.ic.ac.uk/research/crawley/statistics/},
ABSTRACT = {Statistics: An Introduction using R is the first text to offer
such
a concise introduction to a broad array of statistical methods,
at a level that is elementary enough to appeal to a broad range of
disciplines. It is primarily aimed at undergraduate students in
medicine,
engineering, economics and biology – but will also appeal to
postgraduates
who have not previously covered this area, or wish to switch to
using R.}
}
@BOOK{R:Everitt:2005,
AUTHOR = {Brian S. Everitt},
TITLE = {An R and S-Plus® Companion to Multivariate Analysis},
PUBLISHER = {Springer},
YEAR = 2005,
NOTE = {ISBN 1-85233-882-2},
PUBLISHERURL =
{http://www.springeronline.com/sgw/cda/frontpage/0,11855,4-40109-22-34953445-0,00.html},
URL = {http://biostatistics.iop.kcl.ac.uk/publications/everitt/},
ABSTRACT = {In this book the core multivariate methodology is covered along
with some
basic theory for each method described. The necessary R and
S-PLUS code
is given for each analysis in the book, with any differences
between the
two highlighted. A website with all the datasets and code used in
the book
can be found at
http://biostatistics.iop.kcl.ac.uk/publications/everitt/.}
}
@BOOK{R:Harrell:2001,
AUTHOR = {Frank E. Harrell},
TITLE = {Regression Modeling Strategies, with Applications to Linear
Models, Survival Analysis and Logistic Regression},
PUBLISHER = {Springer},
YEAR = 2001,
NOTE = {ISBN 0-387-95232-2},
PUBLISHERURL =
{http://www.springeronline.com/sgw/cda/frontpage/0,11855,4-0-22-2187282-0,00.html},
URL = {http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/RmS},
ABSTRACT = {There are many books that are excellent sources of knowledge
about individual statistical
tools (survival models, general linear models, etc.), but the art
of data analysis is
about choosing and using multiple tools. In the words of
Chatfield "...students typically
know the technical details of regression for example, but not
necessarily when and how to
apply it. This argues the need for a better balance in the
literature and in statistical
teaching between techniques and problem solving strategies."
Whether analyzing risk
factors, adjusting for biases in observational studies, or
developing predictive models,
there are common problems that few regression texts address. For
example, there are
missing data in the majority of datasets one is likely to
encounter (other than those
used in textbooks!) but most regression texts do not include
methods for dealing with
such data effectively, and texts on missing data do not cover
regression modeling.}
}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
