Hi, Am 20.08.2010 um 22:40 schrieb Rahul Nabar:
> What's the typical protocol about the cleanup of /tmp folders? Do > people clean them on each reboot or at intervals with a cron (sounds a > bad idea). I was always under the impression that a reboot cleans them > but apparantly not on my CentOS distro, by default. > > I was burnt earlier today when ompi-ps acted erratically and I > diagnosed it to be caused by stale state information in the /tmp > folder. The remnant of some old dead jobs that had somehow crashed. > > One other option that I've seen mentioned is mounting /tmp on a tmpfs. > Is that a good idea? The risk of using up too much RAM if a program > gets out of hand writing to /tmp. > > On the other hand compute-nodes can go a long time without any > reboots; so a more frequent cleanup cycle on /tmp might be desirable? > I suppose most programs ought to cleanup behind them on /tmp but then > again there are bound to be bad apples. are you using any queuing system? I try to get all applications set up in such a way, that they write all their stuff to $TMPDIR. It's in [OS]GE and I think also in Torque for some time now, to be created automatically (as job specific directory on a node) and removed after the job. A load sensor which checks the space on a node in /scratch and put the queue instance into alarm state, if it falls under a certain value, can in addition prevent a black hole in the cluster, where one after the other job crashes due to missing scratch space. -- Reuti > Any comments? > > -- > Rahul > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
