- R is not multithreaded (or so it was) and thus race condition cannot occur - I would think there is no call to GC at the time of assignment of the return value to a variable. GC is only called within other R calls as R as mentioned above is not multithreaded
Most likely issue is your code itself, out of range indexing, failure to initialise all elements of the allocated structure correctly, 1 and not 0-based indexing, use of other R variables for initialisation that should have been protected but were not etc. Dr Oleg Sklyar Research Technologist AHL / Man Investments Ltd +44 (0)20 7144 3107 oskl...@maninvestments.com > -----Original Message----- > From: r-devel-boun...@r-project.org > [mailto:r-devel-boun...@r-project.org] On Behalf Of Jon Senior > Sent: 27 January 2009 12:09 > To: r-devel@r-project.org > Subject: [Rd] Return values from .Call and garbage collection > > Hi all, > > I'm posting this here as it discusses an issue with an > external C library. If it would be better in R-Help, then I'll repost. > > I'm using an external library which I've written, which > provides a large set of data (>500MB in a highly condensed > format) and the tools to return values from the data. The > functionality has been tested call by call and using valgrind > and works fine, with no memory leaks. After retrieval, I > process the data in R. A specific function is causing a > problem that appears to be related to the garbage collector > (judging by symptoms). > > In the C code, a Matrix is created using > > PROTECT(retVal = allocMatrix(INTSXP, x, y)); > > Values are written into this matrix using > > INTEGER(retVal)[translatedOffset]=z; > > where "translatedOffset" is a conversion from a row/column > pair to an offset as shown in R-exts.pdf. > > The last two lines of the function call are: > > UNPROTECT(1); > return retVal; > > The shared library was compiled with R CMD SHLIB and is > called using .Call. > > Which returns our completed SEXP object to R where processing > continues. > > In R, we continue to process the data, replacing -1s with NAs > (I couldn't find a way to do that in that would make it back > into R), sorting it, and trimming it. All of these operations > are carried out on the original data. > > If I carry out the processing step by step from the > interpreter, everything is fine and the data comes out how I > would expect. But when I run the R code to carry out those > steps, every now and again (Around 1/5th of the time), the > returned data is garbage. I'm expecting to receive a bias per > iteration that should be -5 <= bias <= 5, but for the > garbaged data, I'm getting results of the order of 100s of > thousands out (eg. -220627.7). If I call the routine which > carries out the processing for one iteration from the > intepreter, sometimes I get the correct data, sometimes (with > the same frequency) I get garbage. > > There are two possibilities that I can envisage. > 1) Race condition: R is starting to execute the R code after > the .Call before the .Call has returned, thus the data is corrupted. > 2) Garbage collector: the GC is collecting my data between > the UNPROTECT(1); call and the assignment to an R variable. > > The created matrices can be large (where x > 1000, y > > 100000), but the garbage doesn't appear to be related to the > size of the matrix. > > Any ideas what steps I could take to proceed with this? Or > other possibilities than those I've suggested? For reasons of > confidentiality I'm unable to release test code, and the > large dataset might make testing difficult. > > Thanks in advance > > -- > Jon Senior <j...@restlesslemon.co.uk> > > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > ********************************************************************** Please consider the environment before printing this email or its attachments. The contents of this email are for the named addressees ...{{dropped:19}} ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
