Andre, On May 4, 2012, at 9:50 PM, andre zege wrote:
> I work with problems that have rather large data requirements -- typically > a bunch of multigig arrays. Given how generous R is with using memory, the > only way for me to work with R has been to use bigmatrices from bigmemory > package. One thing that is missing a bit is interoperability of bigmatrices > with C++ and possibly java. What i mean by that is API that would allow > read and write filebacked matrices from C++, and ideally java without being > called from R. Having ability to save armadillo matrices into filebacked > matrices and load them back into armadillo would be another very useful > thing. This would allow really smooth cooperation between various pieces of > software. I would prefer to avoid using Rinside for that. > > I guess i could hack bigmemory C++ code a bit, compile it into a C++ shared > library and it'll do. I guess i could hack it a bit to work with armadillo > matrices as well. I don't want however to reinvent the wheel and if there > is something like that already somewhere i would rather use it for the > moment. Looking very much for suggestions. If there is truly nothing like > that and someone with C++ or especially java development experience is > interested and want to cooperate on this, let me know too. > bigmemory matrices are simply arrays of native types (typically doubles, but bm supports other types, too) so they are trivially readable/writable from both C++ (just read into memory and cast to the array type) and Java (e.g, DoubleBuffer view on a ByteBuffer). So the question is what exactly is the problem? Cheers, Simon > Best > Andre > > NB. I guess something like what i want -- access to the same disc caches > from R, C++, and java (and python) exists in HDF world. I, however, don't > know how performance of HDF compares with bigmemory matrices, which i come > to like and appreciate a lot. If there is someone who could address > simplicity of use and performance of HDF vs bigmemory, it'd be very > interesting. > > [[alternative HTML version deleted]] > > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > > ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
