Re: [Rd] Large vector support in data.frames

2024-07-04 Thread avi.e.gross
integer for efficiency. -Original Message- From: R-devel On Behalf Of Jan van der Laan Sent: Thursday, July 4, 2024 2:38 AM To: r-devel@r-project.org Subject: Re: [Rd] Large vector support in data.frames Ivan, Simon, Thanks for the replies. I can work around the limitation. I currently

Re: [Rd] Large vector support in data.frames

2024-07-03 Thread Jan van der Laan
Ivan, Simon, Thanks for the replies. I can work around the limitation. I currently either divide the data into shards or use a list with (long) vectors depending on what I am trying to do. But I have to transform between the two representations which takes time and memory and often need more

Re: [Rd] Large vector support in data.frames

2024-07-03 Thread Simon Urbanek
The second point is not really an issue - R already uses numerics for larger-than-32-bit indexing at R level and it works just fine for objects up to ca. 72 petabytes. However, the first one is a bit more relevant than one would think. At one point I have experimented with allowing data frames

Re: [Rd] Large vector support in data.frames

2024-07-02 Thread Ivan Krylov via R-devel
В Wed, 19 Jun 2024 09:52:20 +0200 Jan van der Laan пишет: > What is the status of supporting long vectors in data.frames (e.g. > data.frames with more than 2^31 records)? Is this something that is > being worked on? Is there a time line for this? Is this something I > can contribute to? Apolog