[Rd] Slow do.call when having an error
Hi,
We've encountered a difference in running time between a straight function
call and the same call using do.call when the called function generated an
error. We've isolated the problem to the following small reproducible
example:
Consider the following function:
foo <- function(nr = 2e6, nc=3, use.do.call = FALSE) {
nn <- paste("V", 1:nc, sep="")
z <- data.frame(matrix(rnorm(nr*nc), nrow=nr, ncol = nc, dimnames =
list(NULL, nn)))
foo2 <- function(x) x[,"V1"] + x[,"V0"]
if (use.do.call)
do.call(foo2, list(z))
else
foo2(z)
}
foo2, when called, generates an error because it accesses the V0 column
which does not exist. When use.do.call==FALSE, foo2 is called directly.
When use.do.call==TRUE, foo2 is called with the same arguments but using
do.call. Calling foo() takes about 1 second. Calling
foo(use.do.call=TRUE)takes about 20 seconds. Does anybody know what
could explain the difference
in running time? The difference seems to be related to error handling,
since try(foo(use.do.call=TRUE)) takes just 1 second.
We used the latest R version (2.15.0) for the test.
Any insight will be appreciated,
Thanks,
Alon
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Slow do.call when having an error
On 29/05/2012 09:57, Alon Wasserman wrote:
Hi,
We've encountered a difference in running time between a straight function
call and the same call using do.call when the called function generated an
error. We've isolated the problem to the following small reproducible
example:
Consider the following function:
foo<- function(nr = 2e6, nc=3, use.do.call = FALSE) {
nn<- paste("V", 1:nc, sep="")
z<- data.frame(matrix(rnorm(nr*nc), nrow=nr, ncol = nc, dimnames =
list(NULL, nn)))
foo2<- function(x) x[,"V1"] + x[,"V0"]
if (use.do.call)
do.call(foo2, list(z))
else
foo2(z)
}
foo2, when called, generates an error because it accesses the V0 column
which does not exist. When use.do.call==FALSE, foo2 is called directly.
When use.do.call==TRUE, foo2 is called with the same arguments but using
do.call. Calling foo() takes about 1 second. Calling
foo(use.do.call=TRUE)takes about 20 seconds. Does anybody know what
could explain the difference
in running time? The difference seems to be related to error handling,
since try(foo(use.do.call=TRUE)) takes just 1 second.
We used the latest R version (2.15.0) for the test.
Any insight will be appreciated,
Try traceback(max.lines = 10) after each.
With do.call it gives the error equally soon: there is deparsing to be
done in reporting the calls. That is because you passed the object z
and not the symbol z: see the help for do.call.
There are better ways to construct calls: including as here not doing so
but presumably not in the real problem. See e.g. ?call and ?substitute
I don't really see why you did not post this on R-help: it is entirely
about R programming.
Thanks,
Alon
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Out of date instructions to build R using MKL
Yes, these instructions are no longer valid as there has been some reorganization of the mkl libraries. The path /opt/intel/mkl/10.0.3.020/lib/em64t/ is now /opt/intel/mkl/lib/intel64. Also, the only libraries that need to be included are: -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core The trick to getting rid of the "double complex BLAS" error is to use only the gnu compatible libraries (i.e. use mkl_gnu_thread instead of iomp5). I believe the intel-only libraries use a different convention to pass complex numbers between libraries built with fortran? Thankfully this check detects the error instead of crashing at run time. I used the following to build R-2.14.1 with MKL: export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64 ./configure --prefix=/opt/R-2.14.1 --enable-threads=posix --with-lapack --with-blas="-I/opt/intel/mkl/include -L/opt/intel/mkl/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -fopenmp -lpthread -lm" Although a highly tuned BLAS may not help much for many applications, I have seen several orders of magnitude performance improvement in some of my work that uses matrix operations heavily and others in my lab have experienced the same. Having R linked against MKL has been HUGELY important for me personally. A custom tuned ATLAS works well too but I find it frustrating that it needs to be re-tuned for each architecture I use (I tend to distribute jobs in a heterogeneous environment). Thanks! Elliott Forney On Fri, Feb 10, 2012 at 8:15 AM, Milan Bouchet-Valat wrote: > Hi! > > I've been playing with MKL for a few days and I noticed the instructions > in the R Installation Administration manual [1] no longer apply. It > seems that since version 10.0 (the one used by the manual), > libmkl_lapack.so has been renamed/split (although the official > explanations seem to imply this was already the case in 10.0 [2]). > > As a consequence, the instructions for dynamic linking no longer work > with the last version (2011-sp1). This is also the case of what is > explained on several sites like [3] or [4]. The manual's instructions to > link statically to MKL still work fine, though. > > I'm merely signaling this fact to more clued people, since I've not been > able to get R to dynamically link to MKL. I'm always getting this notice > during ./configure: >> checking whether double complex BLAS can be used... no > > Anyways, one of the problems is also that it's no longer possible to > make libRblas.so and libRlapack.so symlinks to the Intel libs, as they > are split into several files. > > If nobody knows how or cares about to fix this ATM, a simple warning > that the instructions are outdated would already improve the situation, > as it took me some time to understand things had changed and I wasn't > just being silly. ;-) > > (That said, I'm not convinced using an external BLAS/LAPACK is really > interesting for standard desktops. Performances gains compared to > default packages are incredible in benchmarks, but for real use cases > multi-threading often makes things slower - at least for me, using gnm. > I guess this is mostly interesting for very larges matrices, and not for > many repeated small operations.) > > Regards > > > 1: http://cran.r-project.org/doc/manuals/R-admin.html#MKL > 2: http://software.intel.com/en-us/forums/showthread.php?t=81302 > 3: > http://www.r-bloggers.com/compiling-64-bit-r-2-10-1-with-mkl-in-linux/ > 4: http://www.rd.dnc.ac.jp/~otsu/lecture/RwithMKL.html > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] virtual superclasses
Dear List,
I'm exploring the use of class inheritance for a package were
developing. I want to try to build on existing a class, i first want to
extend this a bit using a virtual class and then extend this virtual
class in multiple other classes. This would be similar to bar3 in the
example below. Since there are nice functions for creating an object of
the first superclass i would like to use that object for the
initialization but somehow that fails with the error :"node stack
overflow". How could i resolve this problem
Thanks in advance
Bart
> setClass("foo", representation(a="numeric"),prototype = list( a =
integer(0)))
> setClass("bar", representation(b="numeric"))
> setClass("foo2",contains=c("foo") )
> setClass("foo3",contains=c("foo", "VIRTUAL") )
> setClass("foo4",contains=c("foo", "bar") )
> setClass("bar2",contains=c("foo2") )
> setClass("bar3",contains=c("foo3") )
> setClass("bar4",contains=c("foo4") )
> i<-new("foo",a=1)
> new("bar2", i)
An object of class "bar2"
Slot "a":
[1] 1
> new("bar3", i)
Error in tryCatch(expr, error = function(e) { : node stack overflow
> new("bar4", i)
An object of class "bar4"
Slot "a":
[1] 1
Slot "b":
numeric(0)
> sessionInfo()
R version 2.15.0 (2012-03-30)
Platform: x86_64-pc-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=C LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] tools_2.15.0
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
