Re: [Rd] Arrays Partial unserialization

2012-09-09 Thread Wolfgang Huber
Yet another option is provided by the 'rhdf5' package on Bioconductor, which interfaces to HDF5, a cross-platform infrastructure for exactly this purpose: http://www.hdfgroup.org/HDF5/whatishdf5.html Best wishes Wolfgang Aug/31/12 4:44 PM, Gabriel Becker scripsit: I'm not an

Re: [Rd] Arrays Partial unserialization

2012-09-01 Thread Bert Gunter
1. I believe this is an R-Help, not an R-devel topic. Post there. 2. It is not clear to me from your post how the arrays are stored -- as .Rdata files or in some original tabular or database format. I believe the answer would depend on clarifying that point -- or on others understanding what I do

Re: [Rd] Arrays Partial unserialization

2012-08-31 Thread Jeff Ryan
There is no such tool to my knowledge, though the mmap package can do very similar things. In fact, it will be able to do this exactly once I apply a contributed patch to handle endianess. The issue is that rds files are compressed by default, so directly reading requires uncompressing, which mak

Re: [Rd] Arrays Partial unserialization

2012-08-31 Thread Prof Brian Ripley
On 31/08/2012 15:41, Duncan Murdoch wrote: On 31/08/2012 9:47 AM, Damien Georges wrote: Hi all, I'm working with some huge array in R and I need to load several ones to apply some functions that requires to have all my arrays values for each cell... To make it possible, I would like to load on

Re: [Rd] Arrays Partial unserialization

2012-08-31 Thread Gabriel Becker
I'm not an expert in its use, but I believe the bigmemory package offers the functionality you are looking for (or at least similar functionality that can be co-opted for your use-case). See the sub.big.matrix function. Depending on what you mean by "huge" it may offer other benefits as well. HTH,

Re: [Rd] Arrays Partial unserialization

2012-08-31 Thread Duncan Murdoch
On 31/08/2012 9:47 AM, Damien Georges wrote: Hi all, I'm working with some huge array in R and I need to load several ones to apply some functions that requires to have all my arrays values for each cell... To make it possible, I would like to load only a part (for example 100 cells) of all my

[Rd] Arrays Partial unserialization

2012-08-31 Thread Damien Georges
Hi all, I'm working with some huge array in R and I need to load several ones to apply some functions that requires to have all my arrays values for each cell... To make it possible, I would like to load only a part (for example 100 cells) of all my arrays, apply my function, delete all cell