Re: [Rd] modifying large R objects in place

2007-09-29 Thread Petr Savicky
On Fri, Sep 28, 2007 at 08:14:45AM -0500, Luke Tierney wrote: [...] > [...] A related issue is that user-defined > assignment functions always see a NAMED of 2 and hence cannot modify > in place. We've been trying to come up with a reasonable solution to > this, so far without success but I'm moder

Re: [Rd] modifying large R objects in place

2007-09-28 Thread Peter Dalgaard
Duncan Murdoch wrote: > On 9/28/2007 7:45 AM, Petr Savicky wrote: > >> On Fri, Sep 28, 2007 at 12:39:30AM +0200, Peter Dalgaard wrote: >> > ... > >>> Longer-term, I still have some hope for better reference counting, but >>> the semantics of environments make it really ugly -- an envi

Re: [Rd] modifying large R objects in place

2007-09-28 Thread Prof Brian Ripley
On Fri, 28 Sep 2007, Luke Tierney wrote: > On Fri, 28 Sep 2007, Petr Savicky wrote: [...] >> This leads me to a question. Some of the tests, which I did, suggest >> that gc() may not free all the memory, even if I remove all data >> objects by rm() before calling gc(). Is this possible or I must

Re: [Rd] modifying large R objects in place

2007-09-28 Thread Duncan Murdoch
On 9/28/2007 7:45 AM, Petr Savicky wrote: > On Fri, Sep 28, 2007 at 12:39:30AM +0200, Peter Dalgaard wrote: ... >> Longer-term, I still have some hope for better reference counting, but >> the semantics of environments make it really ugly -- an environment can >> contain an object that contains

Re: [Rd] modifying large R objects in place

2007-09-28 Thread Luke Tierney
On Fri, 28 Sep 2007, Petr Savicky wrote: > On Fri, Sep 28, 2007 at 12:39:30AM +0200, Peter Dalgaard wrote: > [...] >>> nrow <- function(...) dim(...)[1] >>> ncol <- function(...) dim(...)[2] >>> >>> At least in my environment, the new versions preserved NAMED == 1. I believe this is a bug in the

Re: [Rd] modifying large R objects in place

2007-09-28 Thread Petr Savicky
On Fri, Sep 28, 2007 at 12:39:30AM +0200, Peter Dalgaard wrote: [...] > >nrow <- function(...) dim(...)[1] > >ncol <- function(...) dim(...)[2] > > > >At least in my environment, the new versions preserved NAMED == 1. > > > Yes, but changing the formal arguments is a bit messy, is it not? Specif

Re: [Rd] modifying large R objects in place

2007-09-27 Thread Peter Dalgaard
Petr Savicky wrote: > Thank you very much for all the explanations. In particular for pointing > out that nrow is not a .Primitive unlike dim, which is the > reason for the difference in their behavior. (I rised the question > of possible bug due to this difference, not just being unsatisfied > wit

Re: [Rd] modifying large R objects in place

2007-09-27 Thread Petr Savicky
Thank you very much for all the explanations. In particular for pointing out that nrow is not a .Primitive unlike dim, which is the reason for the difference in their behavior. (I rised the question of possible bug due to this difference, not just being unsatisfied with nrow). Also, thanks for: On

Re: [Rd] modifying large R objects in place

2007-09-27 Thread Prof Brian Ripley
1) You implicitly coerced 'a' to be numeric and thereby (almost) doubled its size: did you intend to? Does that explain your confusion? 2) I expected NAMED on 'a' to be incremented by nrow(a): here is my understanding. When you called nrow(a) you created another reference to 'a' in the evalu

Re: [Rd] modifying large R objects in place

2007-09-27 Thread Peter Dalgaard
Petr Savicky wrote: > On Wed, Sep 26, 2007 at 10:52:28AM -0700, Byron Ellis wrote: > >> For the most part, doing anything to an R object result in it's >> duplication. You generally have to do a lot of work to NOT copy an R >> object. >> > > Thank you for your response. Unfortunately, you a

Re: [Rd] modifying large R objects in place

2007-09-27 Thread Petr Savicky
In my previous email, I sent the example: > a <- matrix(as.integer(1),nrow=14100,ncol=14100) # 774m > a[1,1] <- 0 # 3.0g > gc() # 1.5g This is misleading. The correct version is a <- matrix(as.integer(1),nrow=14100,ncol=14100) # 774m a[1,1] <- as.integer(0) # 1.5g gc() # 774m So, t

Re: [Rd] modifying large R objects in place

2007-09-27 Thread Petr Savicky
On Wed, Sep 26, 2007 at 10:52:28AM -0700, Byron Ellis wrote: > For the most part, doing anything to an R object result in it's > duplication. You generally have to do a lot of work to NOT copy an R > object. Thank you for your response. Unfortunately, you are right. For example, the allocated memo

Re: [Rd] modifying large R objects in place

2007-09-26 Thread Byron Ellis
For the most part, doing anything to an R object result in it's duplication. You generally have to do a lot of work to NOT copy an R object. On 9/26/07, Petr Savicky <[EMAIL PROTECTED]> wrote: > I have a C function, which performs a transformation > of a large integer matrix. The matrix may be of

[Rd] modifying large R objects in place

2007-09-26 Thread Petr Savicky
I have a C function, which performs a transformation of a large integer matrix. The matrix may be of size 1.6GB, so I can have only one copy in RAM and have to modify it in place. This is possible using .Call and works fine. For debugging, I need two copies of a smaller matrix and modify only one o