Re: [Rd] R/C++/memory leaks
Hi Ross,
On 26 Feb 2007, at 22:34, Ross Boylan wrote:
> On Mon, 2007-02-26 at 16:08 +, Ernest Turro wrote:
>> Thanks for your comments Ross. A couple more comments/queries below:
>>
>> On 26 Feb 2007, at 06:43, Ross Boylan wrote:
>>
>>> [details snipped]
>>>
>>> The use of the R api can be confined to a wrapper function. But
>>> I can
>>> think of no reason that a change to the alternate approach I
>>> outlined
>>> would solve the apparent leaking you describe.
>>>
>>
>> I'm not sure I see how a wrapper function using the R API would
>> suffice. Example:
> It doesn't sound as if it would suffice. I was responding to your
> original remark that
>
>> Since this is a standalone C++ program too, I'd rather use the R API
>> as little as possible... But I will look at your solution if I find
>> it is really necessary.. Thanks
>
> I thought that was expressing a concern about using the alternate
> approach I outlined because it would use the R API. If you need to
> use
> that API for other reasons, you're still stuck with it :)
>>
>> During heavy computation in the C++ function I need to allow
>> interrupts from R. This means that R_CheckUserInterrupt needs to be
>> called during the computation. Therefore, use of the R API can't be
>> confined to just the wrapper function.
>>
>> In fact, I'm worried that some of the libraries I'm using are failing
>> to release memory after interrupt and that that is the problem. I
>> can't see what I could do about that... E.g.
>>
>> #include
>>
>> valarray foo; // I don't know 100% that the foo object hasn't
>> allocated some memory. if the program is interrupted it wouldn't be
>> released
> That's certainly possible, but you seem to be overlooking the
> possibility that all the code is releasing memory appropriately,
> but the
> process's memory footprint isn't going down correspondingly. In my
> experience that's fairly typical behavior.
>
OK, but does this still explain why the footprint keeps increasing
indefinitely when i do run, interrupt, run, interrupt, run,
interrupt..?
> In that case, depending on your point of view, you either don't have a
> problem or you have a hard problem. If you really want the memory
> released back to the system, it's a hard problem. If you don't
> care, as
> long as you have no leaks, all's well.
>
>>
>> I find it's very unfortunate that R_CheckUserInterrupt doesn't return
>> a value. If it did (e.g. if it returned true if an interrupt has
>> occurred), I could just branch off somewhere, clean up properly and
>> return to R.
>>
>> Any ideas on how this could be achieved?
> I can't tell from the info page what function gets called in R if
> there
> is an interrupt, but it sounds as you could do the following hack:
> The R interrupt handler gets a function that calls a C function of
> your
> devising. The C function sets a flag meaning "interrupt requested".
> Then in your main code, you periodically call R_CheckUserInterrupt.
> When it returns you check the flag; if it's set, you cleanup and exit.
> Ross
>
If this is feasible, it's by far the best solution.
in error.c:
void R_CheckUserInterrupt(void)
{
R_CheckStack();
/* This is the point where GUI systems need to do enough event
processing to determine whether there is a user interrupt event
pending. Need to be careful not to do too much event
processing though: if event handlers written in R are allowed
to run at this point then we end up with concurrent R
evaluations and that can cause problems until we have proper
concurrency support. LT */
#if ( defined(HAVE_AQUA) || defined(Win32) )
R_ProcessEvents();
#else
if (R_interrupts_pending)
onintr();
#endif /* Win32 */
}
Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
new function:
int R_CheckInterruptsPending(void)
{
R_CheckStack();
return R_interrupts_pending;
}
and then in my C++ code:
if(R_checkInterruptsPending) {
// clean up
// ...
R_CheckInterruptsPending();
}
R_CheckStack() is declared in R_ext/Utils.h but the variable
R_interrupts_pending isn't, so how could I access it? In other words,
how can I extend error.c .
Thanks,
E
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
[snip]
Sorry. Small mistake fixed below:
>
> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write
> a new function:
>
> int R_CheckInterruptsPending(void)
> {
> R_CheckStack();
> return R_interrupts_pending;
> }
>
> and then in my C++ code:
>
if(R_CheckInterruptsPending) {
// clean up
// ...
R_CheckUserInterrupt();
}
>
> R_CheckStack() is declared in R_ext/Utils.h but the variable
> R_interrupts_pending isn't, so how could I access it? In other
> words, how can I extend error.c .
>
>
> Thanks,
>
> E
>
>
>
>
>
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Checking for user interrupt in a .C() call without without triggering a non-local exit.
On Mon, 26 Feb 2007, Pavel N. Krivitsky wrote: > Hi, > > An R package on which I am working makes a series of very > computationally-intensive and complex .C() calls, that I would like to > make interruptible. However, calling R_CheckUserInterrupt() causes a > non-local exit, so the memory allocated by malloc() is never freed. The > way the code is structured, it might not be practical to replace all the > malloc() calls with R_alloc() calls. > > The question is, can I somehow detect a user interrupt and handle it > gracefully in my own code? A similar question was posed a few months ago > ( http://tolstoy.newcastle.edu.au/R/devel/06/08/6415.html ), but didn't > get any response. Has anything changed since? Not really. There is an internal mechanism for registering C level on.exit routines but this is not in a form that can be made public as it would tie down implementation decisions too much. It is principle possible to build something around R_ToplevelExec, but that is not at this point part of the public API and so is subject to change. We might consider providing something along these lines after 2.5 is released. A tricky issues is that while checking for interrupts is cheap on standard Unix setups where interrupts arrive as signals, it isn't in GUI setups where event processing is needed to detect whether a user interrupt action has occurred (and that checking could cause errors and non-local exits). Best, luke -- Luke Tierney Chair, Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: [EMAIL PROTECTED] Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R/C++/memory leaks
On Tue, 27 Feb 2007, Ernest Turro wrote:
> Hi Ross,
>
> On 26 Feb 2007, at 22:34, Ross Boylan wrote:
>
>> On Mon, 2007-02-26 at 16:08 +, Ernest Turro wrote:
>>> Thanks for your comments Ross. A couple more comments/queries below:
>>>
>>> On 26 Feb 2007, at 06:43, Ross Boylan wrote:
>>>
[details snipped]
The use of the R api can be confined to a wrapper function. But
I can
think of no reason that a change to the alternate approach I
outlined
would solve the apparent leaking you describe.
>>>
>>> I'm not sure I see how a wrapper function using the R API would
>>> suffice. Example:
>> It doesn't sound as if it would suffice. I was responding to your
>> original remark that
>>
>>> Since this is a standalone C++ program too, I'd rather use the R API
>>> as little as possible... But I will look at your solution if I find
>>> it is really necessary.. Thanks
>>
>> I thought that was expressing a concern about using the alternate
>> approach I outlined because it would use the R API. If you need to
>> use
>> that API for other reasons, you're still stuck with it :)
>>>
>>> During heavy computation in the C++ function I need to allow
>>> interrupts from R. This means that R_CheckUserInterrupt needs to be
>>> called during the computation. Therefore, use of the R API can't be
>>> confined to just the wrapper function.
>>>
>>> In fact, I'm worried that some of the libraries I'm using are failing
>>> to release memory after interrupt and that that is the problem. I
>>> can't see what I could do about that... E.g.
>>>
>>> #include
>>>
>>> valarray foo; // I don't know 100% that the foo object hasn't
>>> allocated some memory. if the program is interrupted it wouldn't be
>>> released
>> That's certainly possible, but you seem to be overlooking the
>> possibility that all the code is releasing memory appropriately,
>> but the
>> process's memory footprint isn't going down correspondingly. In my
>> experience that's fairly typical behavior.
>>
>
> OK, but does this still explain why the footprint keeps increasing
> indefinitely when i do run, interrupt, run, interrupt, run,
> interrupt..?
>
>
>> In that case, depending on your point of view, you either don't have a
>> problem or you have a hard problem. If you really want the memory
>> released back to the system, it's a hard problem. If you don't
>> care, as
>> long as you have no leaks, all's well.
>>
>>>
>>> I find it's very unfortunate that R_CheckUserInterrupt doesn't return
>>> a value. If it did (e.g. if it returned true if an interrupt has
>>> occurred), I could just branch off somewhere, clean up properly and
>>> return to R.
>>>
>>> Any ideas on how this could be achieved?
>> I can't tell from the info page what function gets called in R if
>> there
>> is an interrupt, but it sounds as you could do the following hack:
>> The R interrupt handler gets a function that calls a C function of
>> your
>> devising. The C function sets a flag meaning "interrupt requested".
>> Then in your main code, you periodically call R_CheckUserInterrupt.
>> When it returns you check the flag; if it's set, you cleanup and exit.
>> Ross
>>
>
> If this is feasible, it's by far the best solution.
>
> in error.c:
>
> void R_CheckUserInterrupt(void)
> {
> R_CheckStack();
> /* This is the point where GUI systems need to do enough event
>processing to determine whether there is a user interrupt event
>pending. Need to be careful not to do too much event
>processing though: if event handlers written in R are allowed
>to run at this point then we end up with concurrent R
>evaluations and that can cause problems until we have proper
>concurrency support. LT */
> #if ( defined(HAVE_AQUA) || defined(Win32) )
> R_ProcessEvents();
> #else
> if (R_interrupts_pending)
> onintr();
> #endif /* Win32 */
> }
>
> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
> new function:
Unfortunately we can't leave those aside. If standard unix where
interrupts arrive as signals is all you care about then you can just
save, replace and restore the R SIGINT handler around your code with
one that sets a flag of your own. Things are not so simple on GUI
systems where detecting a user interrupt action requires event
processing, which might result in errors and non-local exits in
response to those.
There is an internal mechanism for registering C level on.exit
routines but this is not in a form that can be made public as it would
tie down implementation decisions too much. It is principle possible
to build something around R_ToplevelExec, but that is not at this
point part of the public API and so is subject to change. We might
consider providing a variant of R_CheckInterrupts that either just
checks or that executes cleanup code sometime after 2.5 is released.
Best,
luke
>
> int R_CheckInterruptsPending(void)
> {
> R_CheckStack();
> return R_interrupts_pen
Re: [Rd] R/C++/memory leaks
On 27 Feb 2007, at 17:45, Luke Tierney wrote:
> On Tue, 27 Feb 2007, Ernest Turro wrote:
>
>> Hi Ross,
>>
>> On 26 Feb 2007, at 22:34, Ross Boylan wrote:
>>
>>> On Mon, 2007-02-26 at 16:08 +, Ernest Turro wrote:
Thanks for your comments Ross. A couple more comments/queries
below:
On 26 Feb 2007, at 06:43, Ross Boylan wrote:
> [details snipped]
>
> The use of the R api can be confined to a wrapper function. But
> I can
> think of no reason that a change to the alternate approach I
> outlined
> would solve the apparent leaking you describe.
>
I'm not sure I see how a wrapper function using the R API would
suffice. Example:
>>> It doesn't sound as if it would suffice. I was responding to your
>>> original remark that
>>>
Since this is a standalone C++ program too, I'd rather use the R
API
as little as possible... But I will look at your solution if I find
it is really necessary.. Thanks
>>>
>>> I thought that was expressing a concern about using the alternate
>>> approach I outlined because it would use the R API. If you need to
>>> use
>>> that API for other reasons, you're still stuck with it :)
During heavy computation in the C++ function I need to allow
interrupts from R. This means that R_CheckUserInterrupt needs to be
called during the computation. Therefore, use of the R API can't be
confined to just the wrapper function.
In fact, I'm worried that some of the libraries I'm using are
failing
to release memory after interrupt and that that is the problem. I
can't see what I could do about that... E.g.
#include
valarray foo; // I don't know 100% that the foo object
hasn't
allocated some memory. if the program is interrupted it wouldn't be
released
>>> That's certainly possible, but you seem to be overlooking the
>>> possibility that all the code is releasing memory appropriately,
>>> but the
>>> process's memory footprint isn't going down correspondingly. In my
>>> experience that's fairly typical behavior.
>>>
>>
>> OK, but does this still explain why the footprint keeps increasing
>> indefinitely when i do run, interrupt, run, interrupt, run,
>> interrupt..?
>>
>>
>>> In that case, depending on your point of view, you either don't
>>> have a
>>> problem or you have a hard problem. If you really want the memory
>>> released back to the system, it's a hard problem. If you don't
>>> care, as
>>> long as you have no leaks, all's well.
>>>
I find it's very unfortunate that R_CheckUserInterrupt doesn't
return
a value. If it did (e.g. if it returned true if an interrupt has
occurred), I could just branch off somewhere, clean up properly and
return to R.
Any ideas on how this could be achieved?
>>> I can't tell from the info page what function gets called in R if
>>> there
>>> is an interrupt, but it sounds as you could do the following hack:
>>> The R interrupt handler gets a function that calls a C function of
>>> your
>>> devising. The C function sets a flag meaning "interrupt requested".
>>> Then in your main code, you periodically call R_CheckUserInterrupt.
>>> When it returns you check the flag; if it's set, you cleanup and
>>> exit.
>>> Ross
>>>
>>
>> If this is feasible, it's by far the best solution.
>>
>> in error.c:
>>
>> void R_CheckUserInterrupt(void)
>> {
>> R_CheckStack();
>> /* This is the point where GUI systems need to do enough event
>>processing to determine whether there is a user interrupt
>> event
>>pending. Need to be careful not to do too much event
>>processing though: if event handlers written in R are allowed
>>to run at this point then we end up with concurrent R
>>evaluations and that can cause problems until we have proper
>>concurrency support. LT */
>> #if ( defined(HAVE_AQUA) || defined(Win32) )
>> R_ProcessEvents();
>> #else
>> if (R_interrupts_pending)
>> onintr();
>> #endif /* Win32 */
>> }
>>
>> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
>> new function:
>
> Unfortunately we can't leave those aside. If standard unix where
> interrupts arrive as signals is all you care about then you can just
> save, replace and restore the R SIGINT handler around your code with
> one that sets a flag of your own. Things are not so simple on GUI
> systems where detecting a user interrupt action requires event
> processing, which might result in errors and non-local exits in
> response to those.
>
> There is an internal mechanism for registering C level on.exit
> routines but this is not in a form that can be made public as it would
> tie down implementation decisions too much. It is principle possible
> to build something around R_ToplevelExec, but that is not at this
> point part of the public API and so is subject to change. We might
> conside
Re: [Rd] R/C++/memory leaks
On Tue, 27 Feb 2007, Ernest Turro wrote:
>
> On 27 Feb 2007, at 17:45, Luke Tierney wrote:
>
>> On Tue, 27 Feb 2007, Ernest Turro wrote:
>>
>>> Hi Ross,
>>>
>>> On 26 Feb 2007, at 22:34, Ross Boylan wrote:
>>>
On Mon, 2007-02-26 at 16:08 +, Ernest Turro wrote:
> Thanks for your comments Ross. A couple more comments/queries below:
>
> On 26 Feb 2007, at 06:43, Ross Boylan wrote:
>
>> [details snipped]
>>
>> The use of the R api can be confined to a wrapper function. But
>> I can
>> think of no reason that a change to the alternate approach I
>> outlined
>> would solve the apparent leaking you describe.
>>
>
> I'm not sure I see how a wrapper function using the R API would
> suffice. Example:
It doesn't sound as if it would suffice. I was responding to your
original remark that
> Since this is a standalone C++ program too, I'd rather use the R API
> as little as possible... But I will look at your solution if I find
> it is really necessary.. Thanks
I thought that was expressing a concern about using the alternate
approach I outlined because it would use the R API. If you need to
use
that API for other reasons, you're still stuck with it :)
>
> During heavy computation in the C++ function I need to allow
> interrupts from R. This means that R_CheckUserInterrupt needs to be
> called during the computation. Therefore, use of the R API can't be
> confined to just the wrapper function.
>
> In fact, I'm worried that some of the libraries I'm using are failing
> to release memory after interrupt and that that is the problem. I
> can't see what I could do about that... E.g.
>
> #include
>
> valarray foo; // I don't know 100% that the foo object hasn't
> allocated some memory. if the program is interrupted it wouldn't be
> released
That's certainly possible, but you seem to be overlooking the
possibility that all the code is releasing memory appropriately,
but the
process's memory footprint isn't going down correspondingly. In my
experience that's fairly typical behavior.
>>>
>>> OK, but does this still explain why the footprint keeps increasing
>>> indefinitely when i do run, interrupt, run, interrupt, run,
>>> interrupt..?
>>>
>>>
In that case, depending on your point of view, you either don't have a
problem or you have a hard problem. If you really want the memory
released back to the system, it's a hard problem. If you don't
care, as
long as you have no leaks, all's well.
>
> I find it's very unfortunate that R_CheckUserInterrupt doesn't return
> a value. If it did (e.g. if it returned true if an interrupt has
> occurred), I could just branch off somewhere, clean up properly and
> return to R.
>
> Any ideas on how this could be achieved?
I can't tell from the info page what function gets called in R if
there
is an interrupt, but it sounds as you could do the following hack:
The R interrupt handler gets a function that calls a C function of
your
devising. The C function sets a flag meaning "interrupt requested".
Then in your main code, you periodically call R_CheckUserInterrupt.
When it returns you check the flag; if it's set, you cleanup and exit.
Ross
>>>
>>> If this is feasible, it's by far the best solution.
>>>
>>> in error.c:
>>>
>>> void R_CheckUserInterrupt(void)
>>> {
>>>R_CheckStack();
>>>/* This is the point where GUI systems need to do enough event
>>> processing to determine whether there is a user interrupt event
>>> pending. Need to be careful not to do too much event
>>> processing though: if event handlers written in R are allowed
>>> to run at this point then we end up with concurrent R
>>> evaluations and that can cause problems until we have proper
>>> concurrency support. LT */
>>> #if ( defined(HAVE_AQUA) || defined(Win32) )
>>>R_ProcessEvents();
>>> #else
>>>if (R_interrupts_pending)
>>> onintr();
>>> #endif /* Win32 */
>>> }
>>>
>>> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
>>> new function:
>>
>> Unfortunately we can't leave those aside. If standard unix where
>> interrupts arrive as signals is all you care about then you can just
>> save, replace and restore the R SIGINT handler around your code with
>> one that sets a flag of your own. Things are not so simple on GUI
>> systems where detecting a user interrupt action requires event
>> processing, which might result in errors and non-local exits in
>> response to those.
>>
>> There is an internal mechanism for registering C level on.exit
>> routines but this is not in a form that can be made public as it would
>> tie down implementation decisions too much. It is principle possible
>> to build some
[Rd] asking for advice on how to catch nonexistent list elements
Hi, I am sorry if this is documented or in the archives somewhere, I tried searching and could not find anything. When using the $ operator to get a nonexistent element (or if you prefer, all elements are defined as NULL by default) of a list, it returns NULL, eg > a <- list(maximum=12) > a$maximum [1] 12 > a$maxium # typo NULL I know that this is the documented behavior of $, so this is definitely not a bug. But it leads to bugs that are quite elusive: an error will only be generated when some function eventually chokes on NULL. I would like to know how people cope with that, and if there is a way to generate an error message (at least when debugging code) for the typo above. Thanks, Tamas __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
