Re: Too much memory being used with "find" and parallel

2014-07-25 Thread Martin Møller Skarbiniks Pedersen
On 24 July 2014 19:24, Nelson A. de Oliveira wrote: > On Thu, Jul 24, 2014 at 2:05 PM, Martin Møller Skarbiniks Pedersen > wrote: > > try remove the -j +0 parameter. > > If I am not wrong, "-j +0" should be the same thing as no -j, right? > But just in case, testing without -j the problem is exa

Re: Too much memory being used with "find" and parallel

2014-07-25 Thread Ole Tange
On Thu, Jul 24, 2014 at 4:51 AM, Nelson A. de Oliveira wrote: > find $TILEDIR -type f -name "*.png" | \ > parallel --env optimize -j +0 --eta optimize > > But there is one problem: parallel starts to eat all my RAM and I just > can't run it. >From the man page: --etaShow the

Re: callback on joblog

2014-07-25 Thread Ole Tange
On Wed, Jul 23, 2014 at 10:02 PM, Rob Sargent wrote: > I would like to react to each job's completion. I need to accumulate the > data from each run and decide whether or not it's worth while to continue > with the remaining jobs. I don't think pipe is what I want, but maybe I'm > misunderstanding

Re: callback on joblog

2014-07-25 Thread Rob Sargent
On 07/25/2014 06:27 AM, Ole Tange wrote: On Wed, Jul 23, 2014 at 10:02 PM, Rob Sargent wrote: I would like to react to each job's completion. I need to accumulate the data from each run and decide whether or not it's worth while to continue with the remaining jobs. I don't think pipe is what I

Re: callback on joblog

2014-07-25 Thread Ole Tange
On Fri, Jul 25, 2014 at 5:42 PM, Rob Sargent wrote: > Thanks, yes that might be an option though much must conveyed in "$1". I'm > currently trying a java alternative on this same theme, the meat of which > may morph into "analyze output". You are aware that $1 has a whole family: doit() {

Re: callback on joblog

2014-07-25 Thread Rob Sargent
On 07/25/2014 12:35 PM, Ole Tange wrote: On Fri, Jul 25, 2014 at 5:42 PM, Rob Sargent wrote: Thanks, yes that might be an option though much must conveyed in "$1". I'm currently trying a java alternative on this same theme, the meat of which may morph into "analyze output". You are aware tha