Hi there ...
I vote for changing the first paragraph in OPTIONS:
|
| If command is given, GNU parallel will behave
| similar to xargs.
|
I think this misleading, as in fact it is more like
this:
parallel <--> xargs -n 1
parallel -m <--> xargs
Ole is not sure what'd be the least conf
There is an example for {#} in the manpage. Its description says,
it might be "useful for making input PNG's for ffmpeg":
|
| find . -type f | sort | parallel convert {} {#}.png
|
I'd guess that it wouldn't work with ffmpeg, as the images would
be sorted like this:
10.png 11.png 12.png [...] 1
On Thu, Mar 15, 2012 at 2:30 PM, Thomas Sattler
wrote:
> | If command is given, GNU parallel will behave
> | similar to xargs.
>
> I think this misleading, as in fact it is more like
> this:
>
> parallel <--> xargs -n 1
> parallel -m <--> xargs
>
> Ole is not sure what'd be the least con
Hi again ...
Could someone please enlight me what's the usecase of --delimiter?
I'd guess that values within lines should be split by delim while
input records (lines) are still terminated by \n. But what would
be the difference to --colsep in this case?
If the assumption above is wrong, what do
Hi there ...
eof-strings are only mentioned in the OPTIONS part,
but there aren't any examples for this. Could some-
body please give a real-live usecase for it?
Thomas
Hi there ...
The manpage says, when grouping is disabled, "Output is printed
as soon as possible" and "the outputs from different commands
are mixed together".
As far as I understand, that means not only lines might appear
in quite a funny oder, it might also be, that one line of out-
put is a mi
Hi there ...
I'd vote for improving the manpage about --load:
|
| Only difference is 0 which actually means 0.
|
Question: Does "--load 0" wait until the system is completely
idle (0.00) or just as long as the load drops below 1 (0.x)?
|
| The load average is only sampled every 10 seconds to av
Hi again ...
The idea behind "--load" is great, but I think it's not working that
good. I'd vote for a mechanism of delayed job-starts when "--load"
is in use:
I tried running lrzip in parallel. It's a multithreaded compressor
written by Con Kolivas, optimized for large files. I used "--load"
to
Hi there ...
The subjects line appears three times in the manpage:
"Support for --xargs with --sshlogin is limited and may fail."
"Support for -m with --sshlogin is limited and may fail."
"Support for -X with --sshlogin is limited and may fail."
But what exactly are the limits. What is the pr
(I'm sorry, Ole, for you getting this twice. I made
a mistake and sent my answer to you, not the list.)
>> | If command is given, GNU parallel will behave
>> | similar to xargs.
>
> By not putting any options in we are simply saying parallel
> can solve the same kind of problems as xargs.
So, wh
Now I am using parallel (in fact, sem) to run samtools and other next
generation sequencing analysis.
Some things are quite similar as this blog described:
http://zvfak.blogspot.com/2012/02/samtools-in-parallel.html
But I like use sem in such way:
export PRO="${HOME}/projects/2012-03-09_H3K4me3"
>
On Thu, Mar 15, 2012 at 3:33 PM, Thomas Sattler
wrote:
> Hi there ...
>
> The manpage says, when grouping is disabled, "Output is printed
> as soon as possible" and "the outputs from different commands
> are mixed together".
>
> As far as I understand, that means not only lines might appear
> in q
On Thu, Mar 15, 2012 at 7:10 PM, ningyi shao wrote:
> Now I am using parallel (in fact, sem) to run samtools and other next
> generation sequencing analysis.
> Some things are quite similar as this blog described:
> http://zvfak.blogspot.com/2012/02/samtools-in-parallel.html
> But I like use sem i
On Thu, Mar 15, 2012 at 3:58 PM, Thomas Sattler
wrote:
> The subjects line appears three times in the manpage:
>
> "Support for --xargs with --sshlogin is limited and may fail."
> "Support for -m with --sshlogin is limited and may fail."
> "Support for -X with --sshlogin is limited and may fai
On Thu, Mar 15, 2012 at 3:52 PM, Thomas Sattler
wrote:
> The idea behind "--load" is great, but I think it's not working that
> good. I'd vote for a mechanism of delayed job-starts when "--load"
> is in use.
I can see a delay mechanism can be useful elsewhere, but it is not
that hard for you to
On Thu, Mar 15, 2012 at 3:52 PM, Thomas Sattler
wrote:
> Hi there ...
>
> I'd vote for improving the manpage about --load:
:
> I think the manpage shouldn't leave this as an exercise to the
> users and instead should be modified to answer these questions.
Fixed in git version.
/Ole
Hi again ...
> However, the dir processing is being used by some users so
> removing it as an example does not sound like a good idea.
Maybe I missed the point, but isn't it a fact that -u produces
unreliable output (due to the racing condition) and so the dir
processing will eventually fail?
Th
On Thu, Mar 15, 2012 at 2:50 PM, Thomas Sattler
wrote:
> There is an example for {#} in the manpage. Its description says,
> it might be "useful for making input PNG's for ffmpeg":
>
> |
> | find . -type f | sort | parallel convert {} {#}.png
> |
>
> I'd guess that it wouldn't work with ffmpeg, as
On Thu, Mar 15, 2012 at 9:30 PM, Thomas Sattler
wrote:
> Hi again ...
>
>> However, the dir processing is being used by some users so
>> removing it as an example does not sound like a good idea.
>
> Maybe I missed the point, but isn't it a fact that -u produces
> unreliable output (due to the rac
On Thu, Mar 15, 2012 at 3:20 PM, Thomas Sattler
wrote:
> Hi there ...
>
> eof-strings are only mentioned in the OPTIONS part,
> but there aren't any examples for this. Could some-
> body please give a real-live usecase for it?
I cannot. But the reason why the options exists is because of xargs
co
On Thu, Mar 15, 2012 at 3:05 PM, Thomas Sattler
wrote:
> Hi again ...
>
> Could someone please enlight me what's the usecase of --delimiter?
I cannot. --delimiter is there for xargs compatibility.
The text is heavily inspired by the man page for xargs.
/Ole
In production, we only really have 2 types of sequential formatting.
No leading zeros:
8,9,10,11
and fixed number of digits:
008,009,010,011
'Three digits or more' is not really a case I've seen in the wild.
Just my 2 cents having to work with production image sequences on a
daily basis. I typ
>> The idea behind "--load" is great, but I think it's not working that
>> good. I'd vote for a mechanism of delayed job-starts when "--load"
>> is in use.
>
> I can see a delay mechanism can be useful elsewhere, but it is not
> that hard for you to do yourself:
That was not what I wanted. Yes, a
>> Maybe I missed the point, but isn't it a fact that -u produces
>> unreliable output (due to the racing condition) and so the dir
>> processing will eventually fail?
>
> Not at all. It is just the output to stdout that may be mixed up (with
> -u) or delayed (without -u). Everything else works as
>> eof-strings are only mentioned in the OPTIONS part,
>> but there aren't any examples for this. Could some-
>> body please give a real-live usecase for it?
>
> I cannot. But the reason why the options exists is
> because of xargs compatibility.
OK, so let's forget about the "real-life" aspect.
OK, here is how I (nearly) killed my cluster:
-- Story ---
Trying to see GNU parallel in action, I decided to repack collectl's
logfiles. On my system they grow until about 700-900MB (raw) per day
which becomes about 150MB (gziped).
First I put the
>> Could someone please enlight me what's the usecase of --delimiter?
>
> I cannot. --delimiter is there for xargs compatibility.
>
> The text is heavily inspired by the man page for xargs.
OK, I just put the two manpages next to each other and now
I understand the meaning of "is heavily inspire
> Action speaks louder than words: Instead of voting for it, please do
> the investigation to figure out the limits, and why, how and when it
> fails. Then write your suggestion for that section of the man page.
I see. While reading the manpage I though these were already known
and just not (yet)
On Thu, Mar 15, 2012 at 11:03 PM, Thomas Sattler
wrote:
> Let me change my question: What is, from your point of view,
> the difference between "--colsep" and "--delimiter"?
>
> Or is 'colsep' "just an enhanced version" of 'delimiter',
> that trims the items and uses Perl Regular Expression
> ins
On Thu, Mar 15, 2012 at 9:49 PM, Thomas Sattler
wrote:
> That was not what I wanted. Yes, an experienced user can easily create
> a script that does this, but my intension is something quite different:
>
> GNU parallel does a great job in putting high load on a system. And
> my question was: Shou
On Thu, Mar 15, 2012 at 10:47 PM, Thomas Sattler
wrote:
> OK, here is how I (nearly) killed my cluster:
I like the sound of it: GNU Parallel - the cluster killer!
> As I asked before in "issues with --load": Shouldn't we take
> more care that (not-so-experienced) users do not overload
> their ma
Thomas got me thinking.
One of the problems with --load is that it only limits how many jobs
are started. So you may start way too many. This will give you a load
of 100:
seq 100 | nice parallel -j0 --load 2.00 burnP6
and that is most likely not what you want.
While some programs run multiple
32 matches
Mail list logo
