Hi Kylie, As an alternative in the short term, you could consider deriving from S4Vector's List class, implementing the getListElement() method to lazily create the objects.
Michael On Tue, Jul 23, 2019 at 9:09 AM Bemis, Kylie <k.be...@northeastern.edu> wrote: > > Hello, > > I was wondering if there were any plans for ALTREP lists (VECSXP)? > > It seems to me that they could be supported in a similar way to how ALTSTRING > works, with Elt() and Set_elt() methods, or would there be some problems with > that I’m not seeing due to lists not being atomic vectors? > > I was taking an approach of converting each list element (of a file-based > list data structure) to an ALTREP representation to build up an “ALTREP list”. > > This seems fine for shorter lists with large elements, but I noticed that for > longer lists with smaller elements, this could be far more time-consuming > than simply reading the entire list into memory and returning a non-ALTREP > list: > > > x > <34840 length> matter_list :: out-of-memory list > (1.1 MB real | 543.3 MB virtual) > > > system.time(y <- as.list(x)) > user system elapsed > 1.116 2.175 5.053 > > > system.time(z <- as.altrep(x)) > user system elapsed > 36.295 4.717 41.216 > > > .Internal(inspect(y)) > @108255000 19 VECSXP g1c7 [MARK,NAM(7)] (len=34840, tl=0) > @7f9044d9fc00 14 REALSXP g1c7 [MARK] (len=1129, tl=0) > 404.093,404.096,404.099,404.102,404.105,... > @7f9044d25e00 14 REALSXP g1c7 [MARK] (len=890, tl=0) > 409.924,409.927,409.931,409.934,409.937,... > @7f9044da6000 14 REALSXP g1c7 [MARK] (len=1878, tl=0) > 400.3,400.303,400.306,400.309,400.312,... > @7f9031a6b000 14 REALSXP g1c7 [MARK] (len=2266, tl=0) > 402.179,402.182,402.185,402.188,402.191,... > @7f9031a77a00 14 REALSXP g1c7 [MARK] (len=1981, tl=0) > 403.021,403.024,403.027,403.03,403.033,... > ... > > > .Internal(inspect(z)) > @108210000 19 VECSXP g1c7 [MARK,NAM(7)] (len=34840, tl=0) > @7f904eea7660 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, > len=1129, mem=0) > @7f9050347498 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, len=890, > mem=0) > @7f904d286b20 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, > len=1878, mem=0) > @7f904fd38820 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, > len=2266, mem=0) > @7f904c75ce90 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, > len=1981, mem=0) > ... > > In this situation, it would be much faster and simpler for me to return a > theoretical ALTREP list that serves SEXP elements on-demand, similar to how > ALTSTRING seems to be implemented. > > I don’t know how many other people would get a use out of ALTREP lists, but I > certainly would. > > Are there any plans for this? > > Thanks! > > ~~~ > Kylie Ariel Bemis > Khoury College of Computer Sciences > Northeastern University > kuwisdelu.github.io<https://kuwisdelu.github.io> > > > > > > > > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- Michael Lawrence Scientist, Bioinformatics and Computational Biology Genentech, A Member of the Roche Group Office +1 (650) 225-7760 micha...@gene.com Join Genentech on LinkedIn | Twitter | Facebook | Instagram | YouTube ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
