Re: [Rd] IDE for R C++ package writing ?
First, great thanks to all for all the answers. I confess i was a bit scared about (re)learning a possible tomorrow obsolete tool. I'm however quite astonished nobody proposes another tool. Do 100% R package developers use emacs ? Anyway, given the answers, it seems i'll go on emacs or xemacs. Thanks for the guidance. Vincent __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
Read the help page of gc(). You need to run it with reset=TRUE for the usage to drop back to original. i.e. gc(reset=TRUE). gc() on its own doesn't quite do what you think it would do. Ernest Turro wrote: > Dear all, > > I have wrapped a C++ function in an R package. I allocate/deallocate > memory using C++ 'new' and 'delete'. In order to allow user > interrupts without memory leaks I've moved all the delete statements > required after an interrupt to a separate C++ function freeMemory(), > which is called using on.exit() just before the .C() call. > > I am concerned about the following. In square brackets you see R's > total virtual memory use (VIRT in `top`): > > 1) Load library and data [178MB] (if I run gc(), then [122MB]) > 2) Just before .C [223MB] > 3) Just before freeing memory [325MB] > 4) Just after freeing memory [288MB] > 5) After running gc() [230MB] > > So although the freeMemory function works (frees 37MB), R ends up > using 100MB more after the function call than before it. ls() only > returns the data object so no new objects have been added to the > workspace. > > Do any of you have any idea what could be eating this memory? > > Many thanks, > > Ernest > > PS: it is not practical to use R_alloc et al because C++ allocation/ > deallocation involves constructors/destructors and because the C++ > code is also compiled into a standalone binary (I would rather avoid > maintaining two separate versions). > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] typo in nls.Rd
in R version 2.5.0 Under development (unstable) (2007-02-25 r40804),
the `formula' entry in the `arguments' section of nls.Rd (L25-26) reads
\item{formula}{a nonlinear model \link{formula} including variables and
parameters. Will be coerced to a formula is necessary.}
`is' should be `if'.
Katharine Mullen
Department of Physics and Astronomy
Faculty of Sciences
Vrije Universiteit Amsterdam
de Boelelaan 1081
1081 HV Amsterdam
The Netherlands
room: T.1.06
tel: +31 205987870
fax: +31 205987992
e-mail: [EMAIL PROTECTED]
http://www.nat.vu.nl/~kate/
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
On 26 Feb 2007, at 10:51, Hin-Tak Leung wrote: > Ernest Turro wrote: >> Dear all, >> I have wrapped a C++ function in an R package. I allocate/ >> deallocate memory using C++ 'new' and 'delete'. In order to allow >> user interrupts without memory leaks I've moved all the delete >> statements required after an interrupt to a separate C++ function >> freeMemory(), which is called using on.exit() just before the .C >> () call. >> I am concerned about the following. In square brackets you see >> R's total virtual memory use (VIRT in `top`): >> 1) Load library and data [178MB] (if I run gc(), then [122MB]) >> 2) Just before .C [223MB] >> 3) Just before freeing memory [325MB] >> 4) Just after freeing memory [288MB] >> 5) After running gc() [230MB] >> So although the freeMemory function works (frees 37MB), R ends up >> using 100MB more after the function call than before it. ls() >> only returns the data object so no new objects have been added to >> the workspace. >> Do any of you have any idea what could be eating this memory? >> Many thanks, >> Ernest >> PS: it is not practical to use R_alloc et al because C++ >> allocation/ deallocation involves constructors/destructors and >> because the C++ code is also compiled into a standalone binary (I >> would rather avoid maintaining two separate versions). >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > Read the help page of gc(). You need to run it with reset=TRUE for > the usage to drop back to original. i.e. gc(reset=TRUE). gc() on its > own doesn't quite do what you think it would do. > Thanks, but in this case it barely makes a difference.. :( __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] IDE for R C++ package writing ?
> First, great thanks to all for all the answers. I confess i was a bit > scared about (re)learning a possible tomorrow obsolete tool. > > I'm however quite astonished nobody proposes another tool. Do 100% R > package developers use emacs ? Plenty of folks don't use an IDE at all. Copy/pasting working bits of code from your .Rhistory into a working file is a very useful tactic... --e __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] IDE for R C++ package writing ?
On Monday 26 February 2007 16:51, [EMAIL PROTECTED] wrote:
> > First, great thanks to all for all the answers. I confess i was a bit
> > scared about (re)learning a possible tomorrow obsolete tool.
> >
> > I'm however quite astonished nobody proposes another tool. Do 100% R
> > package developers use emacs ?
>
> Plenty of folks don't use an IDE at all. Copy/pasting working bits of
> code from your .Rhistory into a working file is a very useful tactic...
You kidding, right? (I mean, maybe lots of people do that, but maybe that
ain't such a good idea :-).
R.
P.S. Whether or not emacs + ess + ecb + a whole bunch of other things is or
not a "real IDE" (whatever that means) I think is tangential to the original
question. The issue, if I understand, are editing tools that will make the
editing et al. simpler. So
> > I'm however quite astonished nobody proposes another tool. Do 100% R
> > package developers use emacs ?
>
No. Not 100%. But you said you'll be using Windows but want to move to
GNU/Linux. Then, you might want to use the very same tool over a range of
OSs, or regardless of whether you are in front of your workstation, or
accessing it over a slow modem connection, etc. In such cases, Emacs is an
excellent choice. Or one of the very, very few. In addition, I think you are
seeing an example of "once you try emacs, you often realize that other
choices do not really offer you all that much, but you loose a lot".
HTH,
R.
>
> --e
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Ramón Díaz-Uriarte
Statistical Computing Team
Centro Nacional de Investigaciones Oncológicas (CNIO)
(Spanish National Cancer Center)
Melchor Fernández Almagro, 3
28029 Madrid (Spain)
Fax: +-34-91-224-6972
Phone: +-34-91-224-6900
http://ligarto.org/rdiaz
PGP KeyID: 0xE89B3462
(http://ligarto.org/rdiaz/0xE89B3462.asc)
**NOTA DE CONFIDENCIALIDAD** Este correo electrónico, y en s...{{dropped}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Is there any package on CRAN that uses Fortran-90/95
Hi! Is there any package on CRAN that uses Fortran-90/95? I would like to study it. Thanks, Gregor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] IDE for R C++ package writing ?
Ya, copy/paste from .Rhistory is pretty common. Especially among newbies and oldsters who dislike IDEs. :) [I got burned by Borland, way back when, and basically can't stand "wizards" and the like now...] Looks like someone wrote a "send-to-R" plugin for vim last year: http://www.vim.org/scripts/script.php?script_id=1048 --e On Mon, 26 Feb 2007, Ramon Diaz-Uriarte wrote: > Date: Mon, 26 Feb 2007 16:19:54 +0100 > From: Ramon Diaz-Uriarte <[EMAIL PROTECTED]> > To: r-devel@r-project.org > Cc: [EMAIL PROTECTED], mel <[EMAIL PROTECTED]> > Subject: Re: [Rd] IDE for R C++ package writing ? > > On Monday 26 February 2007 16:51, [EMAIL PROTECTED] wrote: >>> First, great thanks to all for all the answers. I confess i was a bit >>> scared about (re)learning a possible tomorrow obsolete tool. >>> >>> I'm however quite astonished nobody proposes another tool. Do 100% R >>> package developers use emacs ? >> >> Plenty of folks don't use an IDE at all. Copy/pasting working bits of >> code from your .Rhistory into a working file is a very useful tactic... > > You kidding, right? (I mean, maybe lots of people do that, but maybe that > ain't such a good idea :-). > > R. > > P.S. Whether or not emacs + ess + ecb + a whole bunch of other things is or > not a "real IDE" (whatever that means) I think is tangential to the original > question. The issue, if I understand, are editing tools that will make the > editing et al. simpler. So > >>> I'm however quite astonished nobody proposes another tool. Do 100% R >>> package developers use emacs ? >> > > No. Not 100%. But you said you'll be using Windows but want to move to > GNU/Linux. Then, you might want to use the very same tool over a range of > OSs, or regardless of whether you are in front of your workstation, or > accessing it over a slow modem connection, etc. In such cases, Emacs is an > excellent choice. Or one of the very, very few. In addition, I think you are > seeing an example of "once you try emacs, you often realize that other > choices do not really offer you all that much, but you loose a lot". > > HTH, > > R. > > >> >> --e >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
Thanks for your comments Ross. A couple more comments/queries below: On 26 Feb 2007, at 06:43, Ross Boylan wrote: > [details snipped] > > The use of the R api can be confined to a wrapper function. But I can > think of no reason that a change to the alternate approach I outlined > would solve the apparent leaking you describe. > I'm not sure I see how a wrapper function using the R API would suffice. Example: During heavy computation in the C++ function I need to allow interrupts from R. This means that R_CheckUserInterrupt needs to be called during the computation. Therefore, use of the R API can't be confined to just the wrapper function. In fact, I'm worried that some of the libraries I'm using are failing to release memory after interrupt and that that is the problem. I can't see what I could do about that... E.g. #include valarray foo; // I don't know 100% that the foo object hasn't allocated some memory. if the program is interrupted it wouldn't be released I find it's very unfortunate that R_CheckUserInterrupt doesn't return a value. If it did (e.g. if it returned true if an interrupt has occurred), I could just branch off somewhere, clean up properly and return to R. Any ideas on how this could be achieved? Thanks, E >> I am concerned about the following. In square brackets you see R's total virtual memory use (VIRT in `top`): 1) Load library and data [178MB] (if I run gc(), then [122MB]) 2) Just before .C [223MB] 3) Just before freeing memory [325MB] >>> So you explicitly call your freeMemory() function? >> >> This is called thanks to on.exit() >> 4) Just after freeing memory [288MB] >>> There are at least 3 possibilities: >>> * your C++ code is leaking >> >> The number of news and deletes are the same, and so is their >> branching... I don't think it is this. >> >>> * C++ memory is never really returned (Commonly, at least in C, the >>> amount of memory allocated to the process never goes down, even if >>> you do a free. This may depend on the OS and the specific calls >>> the >>> program makes. >> >> OK, but the memory should be freed after the process completes, >> surely? > > Most OS's I know will free memory when a process finishes, except for > shared memory. But is that relevant? I assume the process doesn't > complete until you exit R. Your puzzle seems to involve different > stages within the life of a single process. > >> >>> * You did other stuff in R that's still around. After all you >>> went >>> up +45MB between 1 and 2; maybe it's not so odd you went up +65MB >>> between 2 and 4. >> >> Yep, I do stuff before .C and that accounts for the increase >> before .C. But all the objects created before .C go out of scope by >> 4) and so, after gc(), we should be back to 122MB. As I mentioned, ls >> () after 5) returns only the data loaded in 1). > > In principle (and according to ?on.exit) the expression registered by > on.exit is evaluated when the relevant function is exited. In > principle garbage collection reclaims all unused space (though with no > guarantee of when). > > It may be that the practice is looser than the principle. For > example, > Python always nominally managed memory for you, but I think for > quite awhile it didn't really reclaim the memory (because garbage > collection didn't exist or had been turned off). > > >> 5) After running gc() [230MB] So although the freeMemory function works (frees 37MB), R ends up using 100MB more after the function call than before it. ls() only returns the data object so no new objects have been added to the workspace. Do any of you have any idea what could be eating this memory? Many thanks, Ernest PS: it is not practical to use R_alloc et al because C++ allocation/ deallocation involves constructors/destructors and because the C++ code is also compiled into a standalone binary (I would rather avoid maintaining two separate versions). >>> >>> I use regular C++ new's too (except for the external pointer that's >>> returned). However, you can override the operator new in C++ so >>> that >>> it uses your own allocator, e.g., R_alloc. I'm not sure about >>> all the >>> implications that might make that dangerous (e.g., can the memory be >>> garbage collected? can it be moved?). Overriding new is a bit >>> tricky >>> since there are several variants. In particular, there is one with >>> and one without an exception. Also, invdividual classes can define >>> their own new operators; if you have any, you'd need to change those >>> too. >>> >> >> That sounds rather dangerous! > At least tedious to get right. My statements weren't intended as a > recommendation of this approach; I was just pointing out R_alloc and > C++ allocation could probably be fit together. If your C++ program > isn't doing anything exotic with memory management t
Re: [Rd] Depending on many packages: another best practice question
> You can use
>
> if(require(myPackage)) { ... }
Yes, but the problem with this is that I now have the fact that this
function requires that package stated in two places - in the body of
the function, and in the examples - adding redundancy which makes
maintenance harder.
I guess what I really want, is some way to throw an error that R CMD
check won't complain about - ie. a special dependency not met error.
Hadley
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
Ernest, On Feb 25, 2007, at 12:37 PM, Ernest Turro wrote: > I have wrapped a C++ function in an R package. I allocate/ > deallocate memory using C++ 'new' and 'delete'. In order to allow > user interrupts without memory leaks How do you allow for user interrupts? R doesn't allow interrupts while in .C/.Call on purpose, so it's up to your code to handle interrupts properly. This also implies that you can use the regular methods for error recovery available in your language and handle the interrupt yourself by freeing the memory as needed, your code shouldn't return to R until it has cleaned everything up ... > I've moved all the delete statements required after an interrupt to > a separate C++ function freeMemory(), which is called using on.exit > () just before the .C() call. > This sounds a bit dangerous - how can the separate function know about the previous call and the state it was in before the interrupt (save for hard-wiring everything into one instance)? Again, the crucial part is the interrupt handling in your code - it may fail to release some objects... @Ross [see follow-up]: R_RegisterCFinalizerEx is called on R exit if desired (see arguments). However, I don't think this is relevant to the discussed issue as a process termination frees all its memory. Moreover I don't think Ernest wants to wrap all his classes to R objects - we're not talking about the GC here, it is supposed to be C+ + internal issue. Cheers, Simon > I am concerned about the following. In square brackets you see R's > total virtual memory use (VIRT in `top`): > > 1) Load library and data [178MB] (if I run gc(), then [122MB]) > 2) Just before .C [223MB] > 3) Just before freeing memory [325MB] > 4) Just after freeing memory [288MB] > 5) After running gc() [230MB] > > So although the freeMemory function works (frees 37MB), R ends up > using 100MB more after the function call than before it. ls() only > returns the data object so no new objects have been added to the > workspace. > > Do any of you have any idea what could be eating this memory? > > Many thanks, > > Ernest > > PS: it is not practical to use R_alloc et al because C++ allocation/ > deallocation involves constructors/destructors and because the C++ > code is also compiled into a standalone binary (I would rather avoid > maintaining two separate versions). > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
Hi Simon,
On 26 Feb 2007, at 16:58, Simon Urbanek wrote:
> Ernest,
>
> On Feb 25, 2007, at 12:37 PM, Ernest Turro wrote:
>
>> I have wrapped a C++ function in an R package. I allocate/
>> deallocate memory using C++ 'new' and 'delete'. In order to allow
>> user interrupts without memory leaks
>
> How do you allow for user interrupts? R doesn't allow interrupts
> while in .C/.Call on purpose,
void R_CheckUserInterrupt(void)
> so it's up to your code to handle interrupts properly. This also
> implies that you can use the regular methods for error recovery
> available in your language and handle the interrupt yourself by
> freeing the memory as needed, your code shouldn't return to R until
> it has cleaned everything up ...
How can I detect an interrupt from R inside the C/C++ code without
using the R API? SIGINTs don't get passed on if they come from within
R...
>
>
>> I've moved all the delete statements required after an interrupt
>> to a separate C++ function freeMemory(), which is called using
>> on.exit() just before the .C() call.
>>
>
> This sounds a bit dangerous - how can the separate function know
> about the previous call and the state it was in before the
> interrupt (save for hard-wiring everything into one instance)?
> Again, the crucial part is the interrupt handling in your code - it
> may fail to release some objects...
>
I have to declare variables for any allocated memory globally:
static int foo*;
extern "C"
void func(...) {
foo = new int[1024];
// heavy computation
// inside loop:
R_CheckUserInterrupt()
//end heavy computation
}
extern "C"
void freemem(...) {
delete [] foo;
}
in R:
func <- function() {
on.exit(.C("freemem"))
.C("func")
}
Cheers,
Ernest
> @Ross [see follow-up]: R_RegisterCFinalizerEx is called on R exit
> if desired (see arguments). However, I don't think this is relevant
> to the discussed issue as a process termination frees all its
> memory. Moreover I don't think Ernest wants to wrap all his classes
> to R objects - we're not talking about the GC here, it is supposed
> to be C++ internal issue.
>
> Cheers,
> Simon
>
>
>> I am concerned about the following. In square brackets you see R's
>> total virtual memory use (VIRT in `top`):
>>
>> 1) Load library and data [178MB] (if I run gc(), then [122MB])
>> 2) Just before .C [223MB]
>> 3) Just before freeing memory [325MB]
>> 4) Just after freeing memory [288MB]
>> 5) After running gc() [230MB]
>>
>> So although the freeMemory function works (frees 37MB), R ends up
>> using 100MB more after the function call than before it. ls() only
>> returns the data object so no new objects have been added to the
>> workspace.
>>
>> Do any of you have any idea what could be eating this memory?
>>
>> Many thanks,
>>
>> Ernest
>>
>> PS: it is not practical to use R_alloc et al because C++ allocation/
>> deallocation involves constructors/destructors and because the C++
>> code is also compiled into a standalone binary (I would rather avoid
>> maintaining two separate versions).
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
On Mon, 26 Feb 2007, Hin-Tak Leung wrote: > Read the help page of gc(). You need to run it with reset=TRUE for > the usage to drop back to original. i.e. gc(reset=TRUE). gc() on its > own doesn't quite do what you think it would do. This is almost completely incorrect (apart from the advice to read the help page). The setting for reset= has absolutely no effect on R's memory use. It just clears the internal variable that has kept track of the highest memory use so far. -thomas > Ernest Turro wrote: >> Dear all, >> >> I have wrapped a C++ function in an R package. I allocate/deallocate >> memory using C++ 'new' and 'delete'. In order to allow user >> interrupts without memory leaks I've moved all the delete statements >> required after an interrupt to a separate C++ function freeMemory(), >> which is called using on.exit() just before the .C() call. >> >> I am concerned about the following. In square brackets you see R's >> total virtual memory use (VIRT in `top`): >> >> 1) Load library and data [178MB] (if I run gc(), then [122MB]) >> 2) Just before .C [223MB] >> 3) Just before freeing memory [325MB] >> 4) Just after freeing memory [288MB] >> 5) After running gc() [230MB] >> >> So although the freeMemory function works (frees 37MB), R ends up >> using 100MB more after the function call than before it. ls() only >> returns the data object so no new objects have been added to the >> workspace. >> >> Do any of you have any idea what could be eating this memory? >> >> Many thanks, >> >> Ernest >> >> PS: it is not practical to use R_alloc et al because C++ allocation/ >> deallocation involves constructors/destructors and because the C++ >> code is also compiled into a standalone binary (I would rather avoid >> maintaining two separate versions). >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > Thomas Lumley Assoc. Professor, Biostatistics [EMAIL PROTECTED] University of Washington, Seattle __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
On Mon, 2007-02-26 at 16:08 +, Ernest Turro wrote: > Thanks for your comments Ross. A couple more comments/queries below: > > On 26 Feb 2007, at 06:43, Ross Boylan wrote: > > > [details snipped] > > > > The use of the R api can be confined to a wrapper function. But I can > > think of no reason that a change to the alternate approach I outlined > > would solve the apparent leaking you describe. > > > > I'm not sure I see how a wrapper function using the R API would > suffice. Example: It doesn't sound as if it would suffice. I was responding to your original remark that > Since this is a standalone C++ program too, I'd rather use the R API > as little as possible... But I will look at your solution if I find > it is really necessary.. Thanks I thought that was expressing a concern about using the alternate approach I outlined because it would use the R API. If you need to use that API for other reasons, you're still stuck with it :) > > During heavy computation in the C++ function I need to allow > interrupts from R. This means that R_CheckUserInterrupt needs to be > called during the computation. Therefore, use of the R API can't be > confined to just the wrapper function. > > In fact, I'm worried that some of the libraries I'm using are failing > to release memory after interrupt and that that is the problem. I > can't see what I could do about that... E.g. > > #include > > valarray foo; // I don't know 100% that the foo object hasn't > allocated some memory. if the program is interrupted it wouldn't be > released That's certainly possible, but you seem to be overlooking the possibility that all the code is releasing memory appropriately, but the process's memory footprint isn't going down correspondingly. In my experience that's fairly typical behavior. In that case, depending on your point of view, you either don't have a problem or you have a hard problem. If you really want the memory released back to the system, it's a hard problem. If you don't care, as long as you have no leaks, all's well. > > I find it's very unfortunate that R_CheckUserInterrupt doesn't return > a value. If it did (e.g. if it returned true if an interrupt has > occurred), I could just branch off somewhere, clean up properly and > return to R. > > Any ideas on how this could be achieved? I can't tell from the info page what function gets called in R if there is an interrupt, but it sounds as you could do the following hack: The R interrupt handler gets a function that calls a C function of your devising. The C function sets a flag meaning "interrupt requested". Then in your main code, you periodically call R_CheckUserInterrupt. When it returns you check the flag; if it's set, you cleanup and exit. Ross __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Checking for user interrupt in a .C() call without without triggering a non-local exit.
Hi, An R package on which I am working makes a series of very computationally-intensive and complex .C() calls, that I would like to make interruptible. However, calling R_CheckUserInterrupt() causes a non-local exit, so the memory allocated by malloc() is never freed. The way the code is structured, it might not be practical to replace all the malloc() calls with R_alloc() calls. The question is, can I somehow detect a user interrupt and handle it gracefully in my own code? A similar question was posed a few months ago ( http://tolstoy.newcastle.edu.au/R/devel/06/08/6415.html ), but didn't get any response. Has anything changed since? Thanks, Pavel Krivitsky __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
