Re: [Rd] looking for adice on bigmemory framework with C++ and java interoperability
On 4 May 2012 at 22:31, andre zege wrote: | Simon, thanks for your comment. I guess there is no problem, i am | apparently being lazy/busy and wondered if there is ready code that does | it. You are right, i suppose -- i'll look at the c++ code for bigmatrix and | will try to hack a solution. > You may want to look at the documentation for 'external pointers' in the > "Writing R Extensions" manual, and then consider at Rcpp::XPtr which > > provides > an Rcpp-based route to using external pointers. It's nice having others answering our questions before we can -- many thanks Simon/Dirk! A big.matrix of dimension RxC is a column-major binary file of R*C elements of size 1, 2, 4, or 8 bytes, depending on the type of atomic element. Period, end of story, no header to worry about. So you can use it as you like from any language. Whether you can mmap it conveniently (if needed in shared memory or larger-than-RAM applications) is another story. We make use of the BOOST interprocess library for this. For working in R, the existing R API should be sufficient (though could always be expanded). For working in C++, the C++ API is pretty low-level and of course could benefit from ultimately being Rpp-ified, for example. There are plenty of examples of working in C++ inside bigmemory/biganalytics/bigtabulate. For Java... well, I don't code in Java. You can certainly make use of the data structure easily enough, but whether you can make use of the existing C++ API is something I simply can't answer. I note that one really cool trick is when you have data from another source (e.g. many satellite images) which is already a simple binary file. You can do a trivial hack to create a big.matrix descriptor file, and attach.big.matrix() to it immediately. No traditional read.*() is necessary, and it is super fast. Jay -- John W. Emerson (Jay) Associate Professor of Statistics Department of Statistics Yale University http://www.stat.yale.edu/~jay __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] f951.exe: sorry, unimplemented: 64-bit mode not compiled
On 5/4/2012 6:58 PM, Simon Urbanek wrote: On May 4, 2012, at 8:48 PM, Spencer Graves wrote: Hello: Under my Windows 7 system, "R CMD check DiercxkSpline_1.1-5.tar.gz" fails because: f951.exe: sorry, unimplemented: 64-bit mode not compiled in This typically means that you're using the wrong (old) compiler. The new MinGW compilers support both -m32 and -m64. You have to set the PATH to the new compilers (in the gcc-4.6.3 subdirectory of Rtools) *before* any old compilers in Rtools. Simon: That was it: I had installed several recent versions of Rtools without deleting the old ones and without properly updating the path. I deleted what I had, then reinstalled Rtools215.exe (downloaded yesterday) while being careful to have the installer help me edit the path. The problem disappeared. Thanks very much. Spencer Cheers, Simon make: *** [bispev.o] Error 1 gfortran -m64 -O2 -mtune=core2 -c bispev.f -o bispev.o f951.exe: sorry, unimplemented: 64-bit mode not compiled in make: *** [bispev.o] Error 1 ERROR: compilation failed for package 'DierckxSpline' A similar problem was reported for package "glmnet" (http://stackoverflow.com/questions/10291189/compiling-glmnet-failed-in-windows) plus one with R 2.14.2 (http://r.789695.n4.nabble.com/Problems-when-building-a-package-for-Windows-64-td4464488.html). However, I get this with R 2.15.0 and the latest R tools (reinstalled earlier today). On R-Forge, DierckxSpline has "Build status: Current", which suggests that R-Forge does NOT have this problem. I read through the two replies to these two earlier questions without seeing how to fix my problem. Thanks in advance for any suggestions. Spencer Graves __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Spencer Graves, PE, PhD President and Chief Technology Officer Structure Inspection and Monitoring, Inc. 751 Emerson Ct. San José, CA 95126 ph: 408-655-4567 web: www.structuremonitoring.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] unlist crashes 32-bit R on WinXP when use.names=TRUE
Hi all, I experienced a crash in R-2.15.0 on 32-bit Windows XP (sessionInfo below) when running the piece of code below. I cannot replicate the error on 64-bit Linux, 64-bit Windows, or 32-bit R running under 64-bit Windows. I do not have, and could not find, a 32-bit version of Linux to test this. > NOW <- Sys.time() > FUTURE <- NOW+1:1e7 > crash <- as.character(FUTURE) Error in unlist(unclass(x)[1L:3L]) : promise already under evaluation: recursive default argument reference or earlier problems? > traceback() Error: C stack usage is too close to the limit > # evaluating an expression at this point would cause R to exit ungracefully Here's an example that avoids a lot of unnecessary code: L1 <- list(one=1:1e6, two=1:1e6, three=1:1e6) # no issue with smaller list elements U1 <- unlist(L1, recursive=TRUE, use.names=TRUE) C1 <- c(L1, recursive=TRUE, use.names=TRUE) L2 <- list(one=1:1e7, two=1:1e7, three=1:1e7) # crashes after ~2min with error above U2 <- unlist(L2, recursive=TRUE, use.names=TRUE) C2 <- c(L2, recursive=TRUE, use.names=TRUE) # no issue if use.names=FALSE U3 <- unlist(L2, recursive=TRUE, use.names=FALSE) C3 <- c(L2, recursive=TRUE, use.names=FALSE) I took a look at do_unlist and do_c_dflt in bind.c, but I stopped at NewExtractNames because it is a bit beyond my current understanding. Any thoughts? > sessionInfo() R version 2.15.0 (2012-03-30) Platform: i386-pc-mingw32/i386 (32-bit) locale: [1] LC_COLLATE=English_United States.1252 [2] LC_CTYPE=English_United States.1252 [3] LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base Please let me know if I forgot anything or if there's anything I can do to help. Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com R/Finance 2012: Applied Finance with R www.RinFinance.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
