Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-27 Thread frederik
> > It might still be useful if R could sometimes detect when automated > > cleanup happened and warn the user. Perhaps a simple way could be to > > always create an empty file inside session directory, like > > ".tmp_cleaner_trap". R would never touch this file, but check its > > existence time-to

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-27 Thread frederik
> Frederick posted the way to do this in systems using systemd. We should be > putting that in place, or the equivalent on systems using other tempfile > cleanups. This looks to me like something that "make install" should do, or > perhaps it should be done by people putting together packages for

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Duncan Murdoch
On 26/04/2017 10:39 AM, Tomas Kalibera wrote: I agree this should be solved in configuration of systemd/tmpreaper/whatever tmp cleaner - the cleanup must be prevented in configuration files of these tools. Moving session directories under /var/run (XDG_RUNTIME_DIR) does not seem like a good solu

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread William Dunlap via R-devel
The main point of this discussion has been that deleting the tempdir() a fixed number of days after its creation is a problem. It should be deleted when the process that created it is done. R attempts to do this, but there are cases in which it does not so a backup is needed. Bill Dunlap TIBCO S

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread William Dunlap via R-devel
The Ubuntu machine I use a lot (along with others) must not be cleaning /tmp as it has a fair number of Rtmp* directories in /tmp, even when there are no R sessions running on the machine. I would like to automate their removal but there is no obvious way to see if the R process that created the t

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Dirk Eddelbuettel
On 26 April 2017 at 16:39, Tomas Kalibera wrote: | I agree this should be solved in configuration of | systemd/tmpreaper/whatever tmp cleaner - the cleanup must be prevented Yep. | in configuration files of these tools. Moving session directories under | /var/run (XDG_RUNTIME_DIR) does not se

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Martin Maechler
> Dirk Eddelbuettel > on Wed, 26 Apr 2017 08:40:38 -0500 writes: > On 26 April 2017 at 08:29, Duncan Murdoch wrote: > | This seems like the wrong approach. The problem occurs as soon as the > | tempdir() gets cleaned up: there could be information in temp files > |

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Tomas Kalibera
I agree this should be solved in configuration of systemd/tmpreaper/whatever tmp cleaner - the cleanup must be prevented in configuration files of these tools. Moving session directories under /var/run (XDG_RUNTIME_DIR) does not seem like a good solution to me, sooner or later someone might c

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Dirk Eddelbuettel
On 26 April 2017 at 08:29, Duncan Murdoch wrote: | This seems like the wrong approach. The problem occurs as soon as the | tempdir() gets cleaned up: there could be information in temp files | that gets lost at that point. So the solution should be to prevent the | cleanup, not to continue o

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Duncan Murdoch
On 26/04/2017 4:21 AM, Martin Maechler wrote: on Tue, 25 Apr 2017 21:13:59 -0700 writes: > On Tue, Apr 25, 2017 at 02:41:58PM +, Cook, Malcolm wrote: >> Might this combination serve the purpose: >> * R session keeps an open handle on the tempdir it creates, >> * whate

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-26 Thread Martin Maechler
> > on Tue, 25 Apr 2017 21:13:59 -0700 writes: > On Tue, Apr 25, 2017 at 02:41:58PM +, Cook, Malcolm wrote: >> Might this combination serve the purpose: >> * R session keeps an open handle on the tempdir it creates, >> * whatever tempdir harvesting cron job the

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread frederik
Hi Gabriel, Thanks for asking for a better solution, as far as actually preventing temporary files from getting deleted in the first place. I still don't know very much about other peoples' distributions, but Arch uses Systemd which is the culprit on my system. Systemd's 'tmpfiles.d(5)' man page

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread frederik
On Tue, Apr 25, 2017 at 02:41:58PM +, Cook, Malcolm wrote: > Might this combination serve the purpose: > * R session keeps an open handle on the tempdir it creates, > * whatever tempdir harvesting cron job the user has be made sensitive > enough not to delete open files (includin

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread Cook, Malcolm
> Martin, > > Thanks for your work on this. > > One thing that seems to be missing from the conversation is that recreating > the temp directory will prevent future failures when R wants to write a > temp file, but the files will, of course, not be there. Any code written > assuming the co

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread Gabriel Becker
Martin, Thanks for your work on this. One thing that seems to be missing from the conversation is that recreating the temp directory will prevent future failures when R wants to write a temp file, but the files will, of course, not be there. Any code written assuming the contract is that the temp

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread Martin Maechler
> Jeroen Ooms > on Tue, 25 Apr 2017 15:05:51 +0200 writes: > On Tue, Apr 25, 2017 at 1:00 PM, Martin Maechler > wrote: >> As I've found it is not at all hard to add an option >> which checks the existence and if the directory is no >> longer "valid", tries to recr

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread Cook, Malcolm
Chiming in late on this thread... > > | Are there any packages which > > | would break if a call to 'tempdir' automatically recreated this > > | directory? (Or would it be too much of a performance hit to have > > | 'tempdir' check and even just issue a warning when the director

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread Jeroen Ooms
On Tue, Apr 25, 2017 at 1:00 PM, Martin Maechler wrote: > As I've found it is not at all hard to add an option which > checks the existence and if the directory is no longer "valid", > tries to recreate it (and if it fails doing that it calls the > famous R_Suicide(), as it does when R starts up a

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-25 Thread Martin Maechler
> Dirk Eddelbuettel > on Sun, 23 Apr 2017 09:15:18 -0500 writes: > On 21 April 2017 at 10:34, frede...@ofb.net wrote: > | Hi Mikko, > | > | I was bitten by this recently and I think some of the replies are > | missing the point. As I understand it, the problem con

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-24 Thread Dirk Eddelbuettel
On 24 April 2017 at 12:34, frede...@ofb.net wrote: | As it is, I don't even know what distributions are affected. I'm not | sure how to look up the contents of a "default" configuration on other | distributions. So how do you think R can automate that? Hint: It can't. Dirk -- http://dirk.edde

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-24 Thread frederik
Dirk, Your message felt a bit antagonistic to me, or maybe I'm not understanding what you're trying to say. We all seem to agree that different configurations exist, and that some Linux distributions are configured to delete files in /tmp/ after a certain amount of time (seems to be 10 days for Ar

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-23 Thread Dirk Eddelbuettel
On 21 April 2017 at 10:34, frede...@ofb.net wrote: | Hi Mikko, | | I was bitten by this recently and I think some of the replies are | missing the point. As I understand it, the problem consists of these | elements: | | 1. When R starts, it creates a directory like /tmp/RtmpVIeFj4 | | 2. Right

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread frederik
Hi Mikko, I was bitten by this recently and I think some of the replies are missing the point. As I understand it, the problem consists of these elements: 1. When R starts, it creates a directory like /tmp/RtmpVIeFj4 2. Right after R starts I can create files in this directory with no error

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread Dirk Eddelbuettel
On 21 April 2017 at 15:13, Mikko Korpela wrote: | Despite my obvious failure to read the manual and report this properly, | I will try to make a case. I understand that data stored in a temporary | file may disappear, and for that reason using an alternative TMPDIR | might be advisable. However

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread Mikko Korpela
On 21/04/17 14:42, Joris Meys wrote: In defense of the OP: I would have checked ?tmpdir and missed the information in the manual as well. On the help page there's ample information on the underlying processes that create the dir on multiple platforms. I think adding the last two sentences of prof

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread Mikko Korpela
On 21/04/17 14:03, Prof Brian Ripley wrote: From the R-admin manual §5: 'Various environment variables can be set to determine where R creates its per-session temporary directory. The environment variables TMPDIR, TMP and TEMP are searched in turn and the first one which is set and points to a w

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread Joris Meys
In defense of the OP: I would have checked ?tmpdir and missed the information in the manual as well. On the help page there's ample information on the underlying processes that create the dir on multiple platforms. I think adding the last two sentences of prof. Ripley's quote as a warning to the he

Re: [Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread Prof Brian Ripley
From the R-admin manual §5: 'Various environment variables can be set to determine where R creates its per-session temporary directory. The environment variables TMPDIR, TMP and TEMP are searched in turn and the first one which is set and points to a writable area is used. If none do, the fina

[Rd] tempdir() may be deleted during long-running R session

2017-04-21 Thread Mikko Korpela
Temporary files not accessed for a long time are automatically removed in some Linux distributions and probably other operating systems too, depending on system configuration. This may affect the per-session temporary directory, the path of which is returned by tempdir(). I think it would be ni