Re: [Rd] Large vector support in data.frames

2024-07-03 Thread Simon Urbanek
The second point is not really an issue - R already uses numerics for larger-than-32-bit indexing at R level and it works just fine for objects up to ca. 72 petabytes. However, the first one is a bit more relevant than one would think. At one point I have experimented with allowing data frames

Re: [Rd] Large vector support in data.frames

2024-07-03 Thread Jan van der Laan
Ivan, Simon, Thanks for the replies. I can work around the limitation. I currently either divide the data into shards or use a list with (long) vectors depending on what I am trying to do. But I have to transform between the two representations which takes time and memory and often need more