multi-threaded compiling

[Mischa Baars]

Hi,

I'd like to compile a project of mine using multiple logical cores.

I've attached the problem. It consists of two parts:

1) multi-threaded bash script and / or multi-threaded Makefile

Running bash script functions as expected, but executing the same line of
code with make and / or the command line, does not function. Perhaps
someone could explain to me why?

2) passing a string argument from a bash script and / or Makefile to the
gcc -D option

Running the makefile functions as expected, but I have not been able to get
similar code to work from a bash script. Can someone please explain to me
what I'm doing wrong?

Hope to hear from you soon.

Best regards,
Mischa Baars.


2024031100 - gnu questions.tar.xz
Description: application/xz

Re: "local -g" declaration references local var in enclosing scope

[Adrian Ho]

Thanks much for all the insights, everyone! Indeed, the man page isn't
clear on how to reason about `local -g`. I'm now left with the following
understanding:

1. There is *nothing local* about `local -g`; it operates entirely at the
global level. In particular, it does *not* create a local reference to a
global var, which is the natural assumption given the typical behaviour of
`declare/local`.

2. Because of this, there is no way to *read* a global var if a calling
scope declares a shadowing var, short of unsetting that shadow var.

3. This is NOTABUG.

Have I got that right? If so, can I suggest a documentation enhancement for
`declare`, from:

The `-g` option forces variables to be created or modified at the global
scope, even when `declare` is executed in a shell function.

to:

The `-g` option forces variables to be created or modified at the global
scope, even when `declare` is executed in a shell function, *and no
references to these variables are created in that function's scope*.


Thanks again!

On Mon, Mar 11, 2024 at 12:08 PM Kerin Millar  wrote:

> On Sun, 10 Mar 2024 16:01:10 -0400
> Lawrence Velázquez  wrote:
>
> > On Sun, Mar 10, 2024, at 1:51 PM, Kerin Millar wrote:
> > > Dynamic scoping can be tremendously confusing. The following examples
> > > should help to clarify the present state of affairs.
> > >
> > > $ x() { local a; y; echo "outer: $a"; }
> > > $ y() { local a; a=123; echo "inner: $a"; }
> > > $ x; echo "outermost: $a"
> > > inner: 123
> > > outer:
> > > outermost:
> > >
> > > This is likely as you would expect.
> > >
> > > $ y() { local -g a; a=123; echo "inner: $a"; }
> > > $ x; echo "outermost: $a"
> > > inner: 123
> > > outer: 123
> > > outermost:
> > >
> > > This may not be. There, the effect of the -g option effectively ends
> at
> > > the outermost scope in which the variable, a, was declared. Namely,
> > > that of the x function.
> >
> > This doesn't seem to be accurate; the assignment is performed at
> > the *innermost* declared scope (other than the "local -g" one):
> >
> >   $ x() { local a; y; echo "outer: $a"; }
> >   $ y() { local a; z; echo "inner: $a"; }
> >   $ z() { local -g a; a=123; echo "innermost: $a"; }
> >   $ x; echo "outermost: $a"
> >   innermost: 123
> >   inner: 123
> >   outer:
> >   outermost:
> >
> > Basically, without an assignment, "local -g" does nothing.
>
> It might be tempting to think that the criteria for being "ignored" are
> fulfilled but it is not the case. Below is something that I should also
> have tried before initially posting in this thread.
>
> $ z() { a=123; echo "innermost: $a"; }; unset -v a; x; declare -p a
> innermost: 123
> inner: 123
> outer:
> bash: declare: a: not found
>
> $ z() { local -g a; a=123; echo "innermost: $a"; }; unset -v a; x; declare
> -p a
> innermost: 123
> inner: 123
> outer:
> declare -- a
>
> $ z() { local -g a=456; a=123; echo "innermost: $a"; }; unset -v a; x;
> declare -p a
> innermost: 123
> inner: 123
> outer:
> declare -- a="456"
>
> I think that Greg has it right. The use of the -g option, alone, is
> sufficient to reach into the global scope, though dynamic scoping behaviour
> otherwise remains in effect. That is, one would otherwise still need to pop
> scopes - so to speak - to reach the outermost/bottommost scope.
>
> Speaking of which, to do both of these things has some interesting effects
> ...
>
> $ z() { local -g a; unset -v a; a=123; echo "innermost: $a"; }; unset -v
> a; x; declare -p a
> innermost: 123
> inner: 123
> outer: 123
> declare -- a
>
> $ z() { local -g a; unset -v a; unset -v a; a=123; echo "innermost: $a";
> }; unset -v a; x; declare -p a
> innermost: 123
> inner: 123
> outer: 123
> declare -- a="123"
>
> $ x() { local a; y; local +g a; a=456; echo "outer: $a"; }; unset -v a; x;
> declare -p a
> innermost: 123
> inner: 123
> outer: 456
> declare -- a="123"
>
> I remain somewhat uncertain that the manual conveys enough information to
> be able to perfectly reason with all of this.
>
> --
> Kerin Millar
>


-- 
Regards,
Adrian

Re: %lc/%ls print nothing in C locale if argument has bytes >7f

[Chet Ramey]

On 3/8/24 10:56 AM, Emanuele Torre wrote:

Hello.

I have noticed that, in C locale, %lc prints nothing if the first byte
of the argument is non-ASCII (0x80-0xff).


Only on Linux, and maybe other systems that prefer precomposed unicode
characters. (NFC/NFD is kind of a mess, and has been discussed here
previously.)

The issue is that glibc wc[r]tomb/wcs[r]tombs return -1/EILSEQ when
supplied a wide character in a locale where MB_CUR_MAX == 1. Other
systems (macOS) store a single byte in the returned string. printf
converts the supplied multibyte character string to a wide character
string to get the precision right, then converts it back, which is
how these functions are involved.

Maybe printf should just ignore the `l' modifier and treat %C/%S like
%c/%s if MB_CUR_MAX == 1.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Examples of concurrent coproc usage?

[Zachary Santer]

Was "RFE: enable buffering on null-terminated data"

On Mon, Mar 11, 2024 at 7:54 AM Carl Edquist  wrote:
>
> On Sun, 10 Mar 2024, Zachary Santer wrote:
>
> > On Sun, Mar 10, 2024 at 4:36 PM Carl Edquist  wrote:
> >>
> >> Out of curiosity, do you have an example command line for your use case?
> >
> > My use for 'stdbuf --output=L' is to be able to run a command within a
> > bash coprocess.
>
> Oh, cool, now you're talking!  ;)
>
>
> > (Really, a background process communicating with the parent process
> > through FIFOs, since Bash prints a warning message if you try to run
> > more than one coprocess at a time. Shouldn't make a difference here.)
>
> (Kind of a side-note ... bash's limited coprocess handling was a long
> standing annoyance for me in the past, to the point that I wrote a bash
> coprocess management library to handle multiple active coprocess and give
> convenient methods for interaction.  Perhaps the trickiest bit about
> multiple coprocesses open at once (which I suspect is the reason support
> was never added to bash) is that you don't want the second and subsequent
> coprocesses to inherit the pipe fds of prior open coprocesses.  This can
> result in deadlock if, for instance, you close your write end to coproc1,
> but coproc1 continues to wait for input because coproc2 also has a copy of
> a write end of the pipe to coproc1's input.  So you need to be smart about
> subsequent coprocesses first closing all fds associated with other
> coprocesses.

https://lists.gnu.org/archive/html/help-bash/2021-03/msg00296.html
https://lists.gnu.org/archive/html/help-bash/2021-04/msg00136.html

You're on the money, though there is a preprocessor directive you can
build bash with that will allow it to handle multiple concurrent
coprocesses without complaining: MULTIPLE_COPROCS=1. Chet Ramey's
sticking point was that he hadn't seen coprocesses used enough in the
wild to satisfactorily test that his implementation did in fact keep
the coproc file descriptors out of subshells. If you've got examples
you can direct him to, I'd really appreciate it.

> Word to the wise: you might encounter this issue (coproc2 prevents coproc1
> from seeing its end-of-input) even though you are rigging this up yourself
> with FIFOs rather than bash's coproc builtin.)

In my case, it's mostly a non-issue, because I fork the - now three -
background processes before exec'ing automatic fds redirecting to/from
their FIFO's in the parent process. All the automatic fds get put in
an array, and I do close them all at the beginning of a subsequent
process substitution.

Re: "local -g" declaration references local var in enclosing scope

[Chet Ramey]

On 3/11/24 8:26 AM, Adrian Ho wrote:

Thanks much for all the insights, everyone! Indeed, the man page isn't
clear on how to reason about `local -g`. I'm now left with the following
understanding:

1. There is *nothing local* about `local -g`; it operates entirely at the
global level.


It exists to allow shell functions to create variables at the global scope
with attributes that require the use of `declare', which is not possible
without `-g'.



In particular, it does *not* create a local reference to a
global var, which is the natural assumption given the typical behaviour of
`declare/local`.


I would disagree that this assumption is `natural'.



2. Because of this, there is no way to *read* a global var if a calling
scope declares a shadowing var, short of unsetting that shadow var.


This is how dynamic scoping works.



The `-g` option forces variables to be created or modified at the global
scope, even when `declare` is executed in a shell function.

to:

The `-g` option forces variables to be created or modified at the global
scope, even when `declare` is executed in a shell function, *and no
references to these variables are created in that function's scope*.


I disagree that this particular addition is needed. I'd be happy to
entertain other suggestions about how to clarify the documentation.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: "local -g" declaration references local var in enclosing scope

[Chet Ramey]

On 3/11/24 12:08 AM, Kerin Millar wrote:


Speaking of which, to do both of these things has some interesting effects ...

$ z() { local -g a; unset -v a; a=123; echo "innermost: $a"; }; unset -v a; x; 
declare -p a
innermost: 123
inner: 123
outer: 123
declare -- a

$ z() { local -g a; unset -v a; unset -v a; a=123; echo "innermost: $a"; }; 
unset -v a; x; declare -p a
innermost: 123
inner: 123
outer: 123
declare -- a="123"


These show the normal effects of unset combined with dynamic scoping. This
ability to unset local variables in previous function scopes has been the
subject of, um, spirited discussion.


$ x() { local a; y; local +g a; a=456; echo "outer: $a"; }; unset -v a; x; 
declare -p a
innermost: 123
inner: 123
outer: 456
declare -- a="123"


Let's assume the previous function definitions remain in effect here. In
that case, z continues to unset the local variable definitions in previous
scopes, but the `local +g a' is basically a no-op -- it implies not using
the global scope for a, which is the same as not using the option at all.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: "local -g" declaration references local var in enclosing scope

[Chet Ramey]

On 3/10/24 3:55 PM, Zachary Santer wrote:


Relatedly, how would one set attributes on a variable declared in a
calling function? 'readonly' and 'export' can do it for their
respective attributes, but otherwise, I think you just can't.


You can't (you might be able to using nameref tricks I'm not motivated
enough to investigate); that's a consequence of dynamic scoping and
how `declare' works.

`readonly' and `export' just act on variables found using the usual
dynamic scooping rules. You could write loadable builtins `array',
`assoc', and `integer' to do the same for those attributes if you
desired.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: multi-threaded compiling

[Paul Smith]

On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
> I've attached the problem. It consists of two parts:
> 
> 1) multi-threaded bash script and / or multi-threaded Makefile
> 
> Running bash script functions as expected, but executing the same
> line of code with make and / or the command line, does not function.
> Perhaps someone could explain to me why?
> 
> 2) passing a string argument from a bash script and / or Makefile to
> the gcc -D option
> 
> Running the makefile functions as expected, but I have not been able
> to get similar code to work from a bash script. Can someone please
> explain to me what I'm doing wrong?

I don't understand the problem.  In the third paragraph above you say
the bash script works as expected and the makefile doesn't work, but in
the last paragraph you say that the makefile works as expected but you
can't get it to work in bash.

Please provide actual command invocations (cut and pasted) showing the
output you received and explaining exactly what is wrong with it.

But before you do that, be aware that make does NOT invoke /bin/bash as
its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually an
alias for bash.  On other systems it isn't.

If you want your makefile to always use bash as its shell, you should
add an explicit:

SHELL := /bin/bash

to your makefile to force it.  Maybe that will solve your problem.  If
not we'll need details such as I mention above.

Re: "local -g" declaration references local var in enclosing scope

[Kerin Millar]

On Mon, 11 Mar 2024 11:45:17 -0400
Chet Ramey  wrote:

> On 3/11/24 12:08 AM, Kerin Millar wrote:
> 
> > Speaking of which, to do both of these things has some interesting effects 
> > ...
> > 
> > $ z() { local -g a; unset -v a; a=123; echo "innermost: $a"; }; unset -v a; 
> > x; declare -p a
> > innermost: 123
> > inner: 123
> > outer: 123
> > declare -- a
> > 
> > $ z() { local -g a; unset -v a; unset -v a; a=123; echo "innermost: $a"; }; 
> > unset -v a; x; declare -p a
> > innermost: 123
> > inner: 123
> > outer: 123
> > declare -- a="123"
> 
> These show the normal effects of unset combined with dynamic scoping. This
> ability to unset local variables in previous function scopes has been the
> subject of, um, spirited discussion.

Of course, you are right. I had confused myself into thinking that there was 
some curious interaction between -g and the scope peeling technique on display 
there but I just tested again and could observe no adverse interaction. I'll 
attribute that to only being half-awake at the time.

> 
> > $ x() { local a; y; local +g a; a=456; echo "outer: $a"; }; unset -v a; x; 
> > declare -p a
> > innermost: 123
> > inner: 123
> > outer: 456
> > declare -- a="123"
> 
> Let's assume the previous function definitions remain in effect here. In
> that case, z continues to unset the local variable definitions in previous
> scopes, but the `local +g a' is basically a no-op -- it implies not using
> the global scope for a, which is the same as not using the option at all.

Thanks. To confirm this, I replaced "local +g a" with "local a" and saw that it 
continued to have the same effect of preventing the assignment of 456 from 
happening in the global scope. It came as a minor surprise but does make sense, 
given the purpose of local.

-- 
Kerin Millar

Re: multi-threaded compiling

[Mischa Baars]

Hello Paul,

It seems I'm awake a little longer than you are.

The second paragraph as you see it, belongs to 1)
The fourth paragraph as you see it, belongs to 2)

The actual command invocations (a Makefile, a make.sh script) can be found
in the attachment, as indicated on the first line of the mail. In the
attachment there are two directories, one and two, belonging to 1) and 2)
respectively.

I'm not into Vulcan mindmelds, so I hope everything from the first mail
makes sense to you and everyone on this mailing list now.

Best regards,
Mischa Baars.

On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:

> On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
> > I've attached the problem. It consists of two parts:
> >
> > 1) multi-threaded bash script and / or multi-threaded Makefile
> >
> > Running bash script functions as expected, but executing the same
> > line of code with make and / or the command line, does not function.
> > Perhaps someone could explain to me why?
> >
> > 2) passing a string argument from a bash script and / or Makefile to
> > the gcc -D option
> >
> > Running the makefile functions as expected, but I have not been able
> > to get similar code to work from a bash script. Can someone please
> > explain to me what I'm doing wrong?
>
> I don't understand the problem.  In the third paragraph above you say
> the bash script works as expected and the makefile doesn't work, but in
> the last paragraph you say that the makefile works as expected but you
> can't get it to work in bash.
>
> Please provide actual command invocations (cut and pasted) showing the
> output you received and explaining exactly what is wrong with it.
>
> But before you do that, be aware that make does NOT invoke /bin/bash as
> its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually an
> alias for bash.  On other systems it isn't.
>
> If you want your makefile to always use bash as its shell, you should
> add an explicit:
>
> SHELL := /bin/bash
>
> to your makefile to force it.  Maybe that will solve your problem.  If
> not we'll need details such as I mention above.
>

Re: multi-threaded compiling

[alex xmb sw ratchev]

i also completly dont get ur issue

f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=

  while [[ -v f[++i] ]] ; do
 (( ++r > threads )) &&
wait -n
gcc -c "${f[i]}" &
  done

On Mon, Mar 11, 2024, 18:16 Mischa Baars 
wrote:

> Hello Paul,
>
> It seems I'm awake a little longer than you are.
>
> The second paragraph as you see it, belongs to 1)
> The fourth paragraph as you see it, belongs to 2)
>
> The actual command invocations (a Makefile, a make.sh script) can be found
> in the attachment, as indicated on the first line of the mail. In the
> attachment there are two directories, one and two, belonging to 1) and 2)
> respectively.
>
> I'm not into Vulcan mindmelds, so I hope everything from the first mail
> makes sense to you and everyone on this mailing list now.
>
> Best regards,
> Mischa Baars.
>
> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>
> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
> > > I've attached the problem. It consists of two parts:
> > >
> > > 1) multi-threaded bash script and / or multi-threaded Makefile
> > >
> > > Running bash script functions as expected, but executing the same
> > > line of code with make and / or the command line, does not function.
> > > Perhaps someone could explain to me why?
> > >
> > > 2) passing a string argument from a bash script and / or Makefile to
> > > the gcc -D option
> > >
> > > Running the makefile functions as expected, but I have not been able
> > > to get similar code to work from a bash script. Can someone please
> > > explain to me what I'm doing wrong?
> >
> > I don't understand the problem.  In the third paragraph above you say
> > the bash script works as expected and the makefile doesn't work, but in
> > the last paragraph you say that the makefile works as expected but you
> > can't get it to work in bash.
> >
> > Please provide actual command invocations (cut and pasted) showing the
> > output you received and explaining exactly what is wrong with it.
> >
> > But before you do that, be aware that make does NOT invoke /bin/bash as
> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually an
> > alias for bash.  On other systems it isn't.
> >
> > If you want your makefile to always use bash as its shell, you should
> > add an explicit:
> >
> > SHELL := /bin/bash
> >
> > to your makefile to force it.  Maybe that will solve your problem.  If
> > not we'll need details such as I mention above.
> >
>

Re: multi-threaded compiling

[Paul Smith]

On Mon, 2024-03-11 at 18:14 +0100, Mischa Baars wrote:
> The actual command invocations (a Makefile, a make.sh script) can be
> found in the attachment, as indicated on the first line of the mail.
> In the attachment there are two directories, one and two, belonging
> to 1) and 2) respectively.

That's not what I asked: I asked YOU to run the commands on YOUR system
and cut and paste the results of the run into your email message, with
an explicit explanation of exactly what is wrong with the results and
what you wanted to see instead.  What errors did you get?  Or what was
unexpected about the behavior?

I'm not interested in running random scripts on my system that were
downloaded from an email, or examining their behavior to make sure
they're OK, or setting up a container to run them in, or guessing what
you wanted to happen, etc.

Or maybe someone else will want to do that work instead.

But before putting in more effort on the "misbehaving" makefile, read
this part of my response carefully:

> But before you do that, be aware that make does NOT invoke /bin/bash
> as its shell.  It invokes /bin/sh.  On some systems /bin/sh is
> actually an alias for bash.  On other systems it isn't.
> 
> If you want your makefile to always use bash as its shell, you should
> add an explicit:
> 
> SHELL := /bin/bash
> 
> to your makefile to force it.  Maybe that will solve your problem. 
> If not we'll need details such as I mention above.

and verify in your response you added the suggested line to your
makefile and it didn't help (in the situation where the makefile was
behaving in an unexpected way).

Re: multi-threaded compiling

[Mischa Baars]

Oh, ok. I have executed the examples for you now.

What I expected was also in the first mail:

1) The one.make.sh.log has the expected output. The same commands executed
from a Makefile (one.make.log) do not behave as expected. This also holds
for this command executed from the command line.
2) The two.make.log has the expected. The same commands executed from the
bash script do not behave as expected. I'm having trouble obtaining the
desired results with the bash script.

Regards,
Mischa Baars.

On Mon, Mar 11, 2024 at 6:27 PM Paul Smith  wrote:

> On Mon, 2024-03-11 at 18:14 +0100, Mischa Baars wrote:
> > The actual command invocations (a Makefile, a make.sh script) can be
> > found in the attachment, as indicated on the first line of the mail.
> > In the attachment there are two directories, one and two, belonging
> > to 1) and 2) respectively.
>
> That's not what I asked: I asked YOU to run the commands on YOUR system
> and cut and paste the results of the run into your email message, with
> an explicit explanation of exactly what is wrong with the results and
> what you wanted to see instead.  What errors did you get?  Or what was
> unexpected about the behavior?
>
> I'm not interested in running random scripts on my system that were
> downloaded from an email, or examining their behavior to make sure
> they're OK, or setting up a container to run them in, or guessing what
> you wanted to happen, etc.
>

A container? These are no viri, these are scripts :)


> Or maybe someone else will want to do that work instead.
>
> But before putting in more effort on the "misbehaving" makefile, read
> this part of my response carefully:
>
> > But before you do that, be aware that make does NOT invoke /bin/bash
> > as its shell.  It invokes /bin/sh.  On some systems /bin/sh is
> > actually an alias for bash.  On other systems it isn't.
> >
> > If you want your makefile to always use bash as its shell, you should
> > add an explicit:
> >
> > SHELL := /bin/bash
> >
> > to your makefile to force it.  Maybe that will solve your problem.
> > If not we'll need details such as I mention above.
>
> and verify in your response you added the suggested line to your
> makefile and it didn't help (in the situation where the makefile was
> behaving in an unexpected way).
>

No. Doesn't help the tiniest bit.
SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!}; done; sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]}; e=${?}; echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;
/bin/bash: line 1: wait: 1747087: no such job
  0 pid 1747087 exit 127
/bin/bash: line 1: wait: 1747088: no such job
  1 pid 1747088 exit 127
/bin/bash: line 1: wait: 1747089: no such job
  2 pid 1747089 exit 127
/bin/bash: line 1: wait: 1747090: no such job
  3 pid 1747090 exit 127
/bin/bash: line 1: wait: 1747091: no such job
  4 pid 1747091 exit 127
/bin/bash: line 1: wait: 1747092: no such job
  5 pid 1747092 exit 127
/bin/bash: line 1: wait: 1747093: no such job
  6 pid 1747093 exit 127
/bin/bash: line 1: wait: 1747094: no such job
  7 pid 1747094 exit 127
/bin/bash: line 1: wait: 1747095: no such job
  8 pid 1747095 exit 127
/bin/bash: line 1: wait: 1747096: no such job
  9 pid 1747096 exit 127
/bin/bash: line 1: wait: 1747097: no such job
 10 pid 1747097 exit 127
/bin/bash: line 1: wait: 1747098: no such job
 11 pid 1747098 exit 127
/bin/bash: line 1: wait: 1747099: no such job
 12 pid 1747099 exit 127
/bin/bash: line 1: wait: 1747100: no such job
 13 pid 1747100 exit 127
/bin/bash: line 1: wait: 1747101: no such job
 14 pid 1747101 exit 127
/bin/bash: line 1: wait: 1747102: no such job
 15 pid 1747102 exit 127
/bin/bash: line 1: wait: 1747103: no such job
 16 pid 1747103 exit 127
/bin/bash: line 1: wait: 1747104: no such job
 17 pid 1747104 exit 127
/bin/bash: line 1: wait: 1747105: no such job
 18 pid 1747105 exit 127
/bin/bash: line 1: wait: 1747106: no such job
 19 pid 1747106 exit 127
/bin/bash: line 1: wait: 1747107: no such job
 20 pid 1747107 exit 127
/bin/bash: line 1: wait: 1747108: no such job
 21 pid 1747108 exit 127
/bin/bash: line 1: wait: 1747109: no such job
 22 pid 1747109 exit 127
/bin/bash: line 1: wait: 1747110: no such job
 23 pid 1747110 exit 127
/bin/bash: line 1: wait: 1747111: no such job
 24 pid 1747111 exit 127
/bin/bash: line 1: wait: 1747112: no such job
 25 pid 1747112 exit 127
/bin/bash: line 1: wait: 1747113: no such job
 26 pid 1747113 exit 127
/bin/bash: line 1: wait: 1747114: no such job
 27 pid 1747114 exit 127
/bin/bash: line 1: wait: 1747115: no such job
 28 pid 1747115 exit 127
/bin/bash: line 1: wait: 1747116: no such job
 29 pid 1747116 exit 127
/bin/bash: line 1: wait: 1747117: no such job
 30 pid 1747117 exit 127
/bin/bash: line 1: wait: 1747118: no such job
 31 pid 1747118 exit 127
  0 pid 1746688 exit 0
  1 pid 1746689 exit 1
  2 pid 1746690 exit 2
  3 pid 1746691 exit 3
  4 pid 1746692 exit 4
  5 pid 1746693 exit 5
  6 pid 1746694 exit 6
  7 pid 1746695 exit 

Re: multi-threaded compiling

[Greg Wooledge]

On Mon, Mar 11, 2024 at 06:51:54PM +0100, Mischa Baars wrote:
> SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!}; done; 
> sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]}; e=${?}; 
> echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;
> /bin/bash: line 1: wait: 1747087: no such job
>   0 pid 1747087 exit 127
> /bin/bash: line 1: wait: 1747088: no such job
>   1 pid 1747088 exit 127

Without analyzing this in depth, one thing struck me immediately:
you're using the reserved variable SECONDS, which has special semantics
in bash.  ${SECONDS} is going to expand to an ever-increasing number,
beginning with 5 (since that's the value you assigned), and going up
by 1 per second that the script runs.  I'm assuming that was not your
intention.

In general, any variable whose name is all capital letters *may* have
special meaning to the shell and should not be used for general storage
purposes in scripts.

Re: multi-threaded compiling

[Mischa Baars]

On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
wrote:

> i also completly dont get ur issue
>
> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>
>   while [[ -v f[++i] ]] ; do
>  (( ++r > threads )) &&
> wait -n
> gcc -c "${f[i]}" &
>   done
>

How nice!

wait -n exit 1 & echo $?

You got me the solution :) Except that wait expects a pid after -n.

Maybe

for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;

is what you meant? The equivalence of sequential execution?

First think, then do magic.


>
>

>
> On Mon, Mar 11, 2024, 18:16 Mischa Baars 
> wrote:
>
>> Hello Paul,
>>
>> It seems I'm awake a little longer than you are.
>>
>> The second paragraph as you see it, belongs to 1)
>> The fourth paragraph as you see it, belongs to 2)
>>
>> The actual command invocations (a Makefile, a make.sh script) can be found
>> in the attachment, as indicated on the first line of the mail. In the
>> attachment there are two directories, one and two, belonging to 1) and 2)
>> respectively.
>>
>> I'm not into Vulcan mindmelds, so I hope everything from the first mail
>> makes sense to you and everyone on this mailing list now.
>>
>> Best regards,
>> Mischa Baars.
>>
>> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>>
>> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
>> > > I've attached the problem. It consists of two parts:
>> > >
>> > > 1) multi-threaded bash script and / or multi-threaded Makefile
>> > >
>> > > Running bash script functions as expected, but executing the same
>> > > line of code with make and / or the command line, does not function.
>> > > Perhaps someone could explain to me why?
>> > >
>> > > 2) passing a string argument from a bash script and / or Makefile to
>> > > the gcc -D option
>> > >
>> > > Running the makefile functions as expected, but I have not been able
>> > > to get similar code to work from a bash script. Can someone please
>> > > explain to me what I'm doing wrong?
>> >
>> > I don't understand the problem.  In the third paragraph above you say
>> > the bash script works as expected and the makefile doesn't work, but in
>> > the last paragraph you say that the makefile works as expected but you
>> > can't get it to work in bash.
>> >
>> > Please provide actual command invocations (cut and pasted) showing the
>> > output you received and explaining exactly what is wrong with it.
>> >
>> > But before you do that, be aware that make does NOT invoke /bin/bash as
>> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually an
>> > alias for bash.  On other systems it isn't.
>> >
>> > If you want your makefile to always use bash as its shell, you should
>> > add an explicit:
>> >
>> > SHELL := /bin/bash
>> >
>> > to your makefile to force it.  Maybe that will solve your problem.  If
>> > not we'll need details such as I mention above.
>> >
>>
>

Re: multi-threaded compiling

[alex xmb sw ratchev]

On Mon, Mar 11, 2024, 19:22 Mischa Baars 
wrote:

> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
> wrote:
>
>> i also completly dont get ur issue
>>
>> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>>
>>   while [[ -v f[++i] ]] ; do
>>  (( ++r > threads )) &&
>> wait -n
>> gcc -c "${f[i]}" &
>>   done
>>
>
> How nice!
>
> wait -n exit 1 & echo $?
>

doesnt need a pid
1 : 1 as i wrote it , excepts add 'wait' as new last line

You got me the solution :) Except that wait expects a pid after -n.
>
> Maybe
>
> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>
> is what you meant? The equivalence of sequential execution?
>
> First think, then do magic.
>
>
>>
>>
>
>>
>> On Mon, Mar 11, 2024, 18:16 Mischa Baars 
>> wrote:
>>
>>> Hello Paul,
>>>
>>> It seems I'm awake a little longer than you are.
>>>
>>> The second paragraph as you see it, belongs to 1)
>>> The fourth paragraph as you see it, belongs to 2)
>>>
>>> The actual command invocations (a Makefile, a make.sh script) can be
>>> found
>>> in the attachment, as indicated on the first line of the mail. In the
>>> attachment there are two directories, one and two, belonging to 1) and 2)
>>> respectively.
>>>
>>> I'm not into Vulcan mindmelds, so I hope everything from the first mail
>>> makes sense to you and everyone on this mailing list now.
>>>
>>> Best regards,
>>> Mischa Baars.
>>>
>>> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>>>
>>> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
>>> > > I've attached the problem. It consists of two parts:
>>> > >
>>> > > 1) multi-threaded bash script and / or multi-threaded Makefile
>>> > >
>>> > > Running bash script functions as expected, but executing the same
>>> > > line of code with make and / or the command line, does not function.
>>> > > Perhaps someone could explain to me why?
>>> > >
>>> > > 2) passing a string argument from a bash script and / or Makefile to
>>> > > the gcc -D option
>>> > >
>>> > > Running the makefile functions as expected, but I have not been able
>>> > > to get similar code to work from a bash script. Can someone please
>>> > > explain to me what I'm doing wrong?
>>> >
>>> > I don't understand the problem.  In the third paragraph above you say
>>> > the bash script works as expected and the makefile doesn't work, but in
>>> > the last paragraph you say that the makefile works as expected but you
>>> > can't get it to work in bash.
>>> >
>>> > Please provide actual command invocations (cut and pasted) showing the
>>> > output you received and explaining exactly what is wrong with it.
>>> >
>>> > But before you do that, be aware that make does NOT invoke /bin/bash as
>>> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually an
>>> > alias for bash.  On other systems it isn't.
>>> >
>>> > If you want your makefile to always use bash as its shell, you should
>>> > add an explicit:
>>> >
>>> > SHELL := /bin/bash
>>> >
>>> > to your makefile to force it.  Maybe that will solve your problem.  If
>>> > not we'll need details such as I mention above.
>>> >
>>>
>>

Re: multi-threaded compiling

[Mischa Baars]

Hi Greg,

Good point. One for you :)

Cheerz,
Mischa.

On Mon, Mar 11, 2024 at 7:14 PM Greg Wooledge  wrote:

> On Mon, Mar 11, 2024 at 06:51:54PM +0100, Mischa Baars wrote:
> > SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!};
> done; sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]};
> e=${?}; echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;
> > /bin/bash: line 1: wait: 1747087: no such job
> >   0 pid 1747087 exit 127
> > /bin/bash: line 1: wait: 1747088: no such job
> >   1 pid 1747088 exit 127
>
> Without analyzing this in depth, one thing struck me immediately:
> you're using the reserved variable SECONDS, which has special semantics
> in bash.  ${SECONDS} is going to expand to an ever-increasing number,
> beginning with 5 (since that's the value you assigned), and going up
> by 1 per second that the script runs.  I'm assuming that was not your
> intention.
>
> In general, any variable whose name is all capital letters *may* have
> special meaning to the shell and should not be used for general storage
> purposes in scripts.
>

Re: multi-threaded compiling

[Mischa Baars]

On Mon, Mar 11, 2024 at 7:14 PM Greg Wooledge  wrote:

> On Mon, Mar 11, 2024 at 06:51:54PM +0100, Mischa Baars wrote:
> > SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!};
> done; sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]};
> e=${?}; echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;
> > /bin/bash: line 1: wait: 1747087: no such job
> >   0 pid 1747087 exit 127
> > /bin/bash: line 1: wait: 1747088: no such job
> >   1 pid 1747088 exit 127
>
> Without analyzing this in depth, one thing struck me immediately:
> you're using the reserved variable SECONDS, which has special semantics
> in bash.  ${SECONDS} is going to expand to an ever-increasing number,
> beginning with 5 (since that's the value you assigned), and going up
> by 1 per second that the script runs.  I'm assuming that was not your
> intention.
>

It wasn't. I modified the script and makefile accordingly.


>
> In general, any variable whose name is all capital letters *may* have
> special meaning to the shell and should not be used for general storage
> purposes in scripts.
>

Completely right again. I will definitely use these recommendations next
time I write a script or a makefile.

Re: multi-threaded compiling

[Greg Wooledge]

On Mon, Mar 11, 2024 at 07:22:39PM +0100, Mischa Baars wrote:
> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
> wrote:
> 
> > i also completly dont get ur issue
> >
> > f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
> >
> >   while [[ -v f[++i] ]] ; do
> >  (( ++r > threads )) &&
> > wait -n
> > gcc -c "${f[i]}" &
> >   done
> >
> 
> How nice!
> 
> wait -n exit 1 & echo $?
> 
> You got me the solution :) Except that wait expects a pid after -n.

No, wait -n without a PID waits for any one job to complete.

hobbit:~$ sleep 123 & sleep 234 & sleep 3 & wait -n
[1] 513337
[2] 513338
[3] 513339
[3]+  Donesleep 3
hobbit:~$ ps
PID TTY  TIME CMD
   1138 pts/000:00:00 bash
 513337 pts/000:00:00 sleep
 513338 pts/000:00:00 sleep
 513341 pts/000:00:00 ps
hobbit:~$ 

Re: multi-threaded compiling

[alex xmb sw ratchev]

On Mon, Mar 11, 2024, 19:25 alex xmb sw ratchev  wrote:

>
>
> On Mon, Mar 11, 2024, 19:22 Mischa Baars 
> wrote:
>
>> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
>> wrote:
>>
>>> i also completly dont get ur issue
>>>
>>> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>>>
>>>   while [[ -v f[++i] ]] ; do
>>>  (( ++r > threads )) &&
>>> wait -n
>>> gcc -c "${f[i]}" &
>>>   done
>>>
>>
>> How nice!
>>
>> wait -n exit 1 & echo $?
>>
>
> doesnt need a pid
> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>
> You got me the solution :) Except that wait expects a pid after -n.
>>
>> Maybe
>>
>> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>>
>
maxt=32
for (( .. )) ; do
(( i > maxt )) && wait -n
"$process_here" &
done

wait

eof

is what you meant? The equivalence of sequential execution?
>>
>> First think, then do magic.
>>
>>
>>>
>>>
>>
>>>
>>> On Mon, Mar 11, 2024, 18:16 Mischa Baars 
>>> wrote:
>>>
 Hello Paul,

 It seems I'm awake a little longer than you are.

 The second paragraph as you see it, belongs to 1)
 The fourth paragraph as you see it, belongs to 2)

 The actual command invocations (a Makefile, a make.sh script) can be
 found
 in the attachment, as indicated on the first line of the mail. In the
 attachment there are two directories, one and two, belonging to 1) and
 2)
 respectively.

 I'm not into Vulcan mindmelds, so I hope everything from the first mail
 makes sense to you and everyone on this mailing list now.

 Best regards,
 Mischa Baars.

 On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:

 > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
 > > I've attached the problem. It consists of two parts:
 > >
 > > 1) multi-threaded bash script and / or multi-threaded Makefile
 > >
 > > Running bash script functions as expected, but executing the same
 > > line of code with make and / or the command line, does not function.
 > > Perhaps someone could explain to me why?
 > >
 > > 2) passing a string argument from a bash script and / or Makefile to
 > > the gcc -D option
 > >
 > > Running the makefile functions as expected, but I have not been able
 > > to get similar code to work from a bash script. Can someone please
 > > explain to me what I'm doing wrong?
 >
 > I don't understand the problem.  In the third paragraph above you say
 > the bash script works as expected and the makefile doesn't work, but
 in
 > the last paragraph you say that the makefile works as expected but you
 > can't get it to work in bash.
 >
 > Please provide actual command invocations (cut and pasted) showing the
 > output you received and explaining exactly what is wrong with it.
 >
 > But before you do that, be aware that make does NOT invoke /bin/bash
 as
 > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually
 an
 > alias for bash.  On other systems it isn't.
 >
 > If you want your makefile to always use bash as its shell, you should
 > add an explicit:
 >
 > SHELL := /bin/bash
 >
 > to your makefile to force it.  Maybe that will solve your problem.  If
 > not we'll need details such as I mention above.
 >

>>>

Re: multi-threaded compiling

[alex xmb sw ratchev]

other output

~/2024031100 - gnu questions/one $ make
SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!};
done; sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]};
e=${?}; echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;
/data/data/com.termux/files/usr/bin/sh: 1: Syntax error: Bad for loop
variable
make: *** [Makefile:3: all] Error 2
~/2024031100 - gnu questions/one $ cd ..
~/2024031100 - gnu questions $ cd two
~/2024031100 - gnu questions/two $ make
STRINGIZED: 0, STRING: one and two
STRINGIZED: 0, STRING: one and two
STRINGIZED: 1, STRING: one and two
STRINGIZED: 1, STRING: one and two
STRINGIZED: 0, STRING: one and two
STRINGIZED: 0, STRING: one and two
STRINGIZED: 1, STRING: one and two
STRINGIZED: 1, STRING: one and two
STRINGIZED: 0, STRING: one and two
STRINGIZED: 0, STRING: one and two
STRINGIZED: 1, STRING: one and two
STRINGIZED: 1, STRING: one and two

On Mon, Mar 11, 2024, 18:54 Mischa Baars 
wrote:

> Oh, ok. I have executed the examples for you now.
>
> What I expected was also in the first mail:
>
> 1) The one.make.sh.log has the expected output. The same commands executed
> from a Makefile (one.make.log) do not behave as expected. This also holds
> for this command executed from the command line.
> 2) The two.make.log has the expected. The same commands executed from the
> bash script do not behave as expected. I'm having trouble obtaining the
> desired results with the bash script.
>
> Regards,
> Mischa Baars.
>
> On Mon, Mar 11, 2024 at 6:27 PM Paul Smith  wrote:
>
> > On Mon, 2024-03-11 at 18:14 +0100, Mischa Baars wrote:
> > > The actual command invocations (a Makefile, a make.sh script) can be
> > > found in the attachment, as indicated on the first line of the mail.
> > > In the attachment there are two directories, one and two, belonging
> > > to 1) and 2) respectively.
> >
> > That's not what I asked: I asked YOU to run the commands on YOUR system
> > and cut and paste the results of the run into your email message, with
> > an explicit explanation of exactly what is wrong with the results and
> > what you wanted to see instead.  What errors did you get?  Or what was
> > unexpected about the behavior?
> >
> > I'm not interested in running random scripts on my system that were
> > downloaded from an email, or examining their behavior to make sure
> > they're OK, or setting up a container to run them in, or guessing what
> > you wanted to happen, etc.
> >
>
> A container? These are no viri, these are scripts :)
>
>
> > Or maybe someone else will want to do that work instead.
> >
> > But before putting in more effort on the "misbehaving" makefile, read
> > this part of my response carefully:
> >
> > > But before you do that, be aware that make does NOT invoke /bin/bash
> > > as its shell.  It invokes /bin/sh.  On some systems /bin/sh is
> > > actually an alias for bash.  On other systems it isn't.
> > >
> > > If you want your makefile to always use bash as its shell, you should
> > > add an explicit:
> > >
> > > SHELL := /bin/bash
> > >
> > > to your makefile to force it.  Maybe that will solve your problem.
> > > If not we'll need details such as I mention above.
> >
> > and verify in your response you added the suggested line to your
> > makefile and it didn't help (in the situation where the makefile was
> > behaving in an unexpected way).
> >
>
> No. Doesn't help the tiniest bit.
>

Re: multi-threaded compiling

[Paul Smith]

On Mon, 2024-03-11 at 19:37 +0100, alex xmb sw ratchev wrote:
> /data/data/com.termux/files/usr/bin/sh: 1: Syntax error: Bad for loop
> variable

This is because of the issue I mentioned in my initial reply.  This
invocation is using /bin/sh which is a POSIX shell but is not bash;
probably it's dash or similar (you are running on Debian or one of the
GNU/Linux distros that build on it).

If you set "SHELL := /bin/bash" (or whatever the path is for bash on
your system) in the makefile you won't see this, I expect.

Re: multi-threaded compiling

[Mischa Baars]

You mean:

for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;

with one command and one wait in a single loop. And this does execute on
the command line. How interesting!

for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
wait -n; echo $?; done;

Because this doesn't and to be honest, I needed the pid and its index to
retrieve gcc's output from a log file array afterwards.

On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
wrote:

>
>
> On Mon, Mar 11, 2024, 19:22 Mischa Baars 
> wrote:
>
>> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
>> wrote:
>>
>>> i also completly dont get ur issue
>>>
>>> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>>>
>>>   while [[ -v f[++i] ]] ; do
>>>  (( ++r > threads )) &&
>>> wait -n
>>> gcc -c "${f[i]}" &
>>>   done
>>>
>>
>> How nice!
>>
>> wait -n exit 1 & echo $?
>>
>
> doesnt need a pid
> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>
> You got me the solution :) Except that wait expects a pid after -n.
>>
>> Maybe
>>
>> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>>
>> is what you meant? The equivalence of sequential execution?
>>
>> First think, then do magic.
>>
>>
>>>
>>>
>>
>>>
>>> On Mon, Mar 11, 2024, 18:16 Mischa Baars 
>>> wrote:
>>>
 Hello Paul,

 It seems I'm awake a little longer than you are.

 The second paragraph as you see it, belongs to 1)
 The fourth paragraph as you see it, belongs to 2)

 The actual command invocations (a Makefile, a make.sh script) can be
 found
 in the attachment, as indicated on the first line of the mail. In the
 attachment there are two directories, one and two, belonging to 1) and
 2)
 respectively.

 I'm not into Vulcan mindmelds, so I hope everything from the first mail
 makes sense to you and everyone on this mailing list now.

 Best regards,
 Mischa Baars.

 On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:

 > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
 > > I've attached the problem. It consists of two parts:
 > >
 > > 1) multi-threaded bash script and / or multi-threaded Makefile
 > >
 > > Running bash script functions as expected, but executing the same
 > > line of code with make and / or the command line, does not function.
 > > Perhaps someone could explain to me why?
 > >
 > > 2) passing a string argument from a bash script and / or Makefile to
 > > the gcc -D option
 > >
 > > Running the makefile functions as expected, but I have not been able
 > > to get similar code to work from a bash script. Can someone please
 > > explain to me what I'm doing wrong?
 >
 > I don't understand the problem.  In the third paragraph above you say
 > the bash script works as expected and the makefile doesn't work, but
 in
 > the last paragraph you say that the makefile works as expected but you
 > can't get it to work in bash.
 >
 > Please provide actual command invocations (cut and pasted) showing the
 > output you received and explaining exactly what is wrong with it.
 >
 > But before you do that, be aware that make does NOT invoke /bin/bash
 as
 > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually
 an
 > alias for bash.  On other systems it isn't.
 >
 > If you want your makefile to always use bash as its shell, you should
 > add an explicit:
 >
 > SHELL := /bin/bash
 >
 > to your makefile to force it.  Maybe that will solve your problem.  If
 > not we'll need details such as I mention above.
 >

>>>

Re: multi-threaded compiling

[Mischa Baars]

Which sort of brings us back to the original question I suppose. Who does
that line of code function from a script and why does it fail from the
command line? My guess was that the same thing makes this line fail from
the Makefile.

On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars 
wrote:

> You mean:
>
> for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;
>
> with one command and one wait in a single loop. And this does execute on
> the command line. How interesting!
>
> for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
> wait -n; echo $?; done;
>
> Because this doesn't and to be honest, I needed the pid and its index to
> retrieve gcc's output from a log file array afterwards.
>
> On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
> wrote:
>
>>
>>
>> On Mon, Mar 11, 2024, 19:22 Mischa Baars 
>> wrote:
>>
>>> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
>>> wrote:
>>>
 i also completly dont get ur issue

 f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=

   while [[ -v f[++i] ]] ; do
  (( ++r > threads )) &&
 wait -n
 gcc -c "${f[i]}" &
   done

>>>
>>> How nice!
>>>
>>> wait -n exit 1 & echo $?
>>>
>>
>> doesnt need a pid
>> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>>
>> You got me the solution :) Except that wait expects a pid after -n.
>>>
>>> Maybe
>>>
>>> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>>>
>>> is what you meant? The equivalence of sequential execution?
>>>
>>> First think, then do magic.
>>>
>>>


>>>

 On Mon, Mar 11, 2024, 18:16 Mischa Baars 
 wrote:

> Hello Paul,
>
> It seems I'm awake a little longer than you are.
>
> The second paragraph as you see it, belongs to 1)
> The fourth paragraph as you see it, belongs to 2)
>
> The actual command invocations (a Makefile, a make.sh script) can be
> found
> in the attachment, as indicated on the first line of the mail. In the
> attachment there are two directories, one and two, belonging to 1) and
> 2)
> respectively.
>
> I'm not into Vulcan mindmelds, so I hope everything from the first mail
> makes sense to you and everyone on this mailing list now.
>
> Best regards,
> Mischa Baars.
>
> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>
> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
> > > I've attached the problem. It consists of two parts:
> > >
> > > 1) multi-threaded bash script and / or multi-threaded Makefile
> > >
> > > Running bash script functions as expected, but executing the same
> > > line of code with make and / or the command line, does not
> function.
> > > Perhaps someone could explain to me why?
> > >
> > > 2) passing a string argument from a bash script and / or Makefile
> to
> > > the gcc -D option
> > >
> > > Running the makefile functions as expected, but I have not been
> able
> > > to get similar code to work from a bash script. Can someone please
> > > explain to me what I'm doing wrong?
> >
> > I don't understand the problem.  In the third paragraph above you say
> > the bash script works as expected and the makefile doesn't work, but
> in
> > the last paragraph you say that the makefile works as expected but
> you
> > can't get it to work in bash.
> >
> > Please provide actual command invocations (cut and pasted) showing
> the
> > output you received and explaining exactly what is wrong with it.
> >
> > But before you do that, be aware that make does NOT invoke /bin/bash
> as
> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually
> an
> > alias for bash.  On other systems it isn't.
> >
> > If you want your makefile to always use bash as its shell, you should
> > add an explicit:
> >
> > SHELL := /bin/bash
> >
> > to your makefile to force it.  Maybe that will solve your problem.
> If
> > not we'll need details such as I mention above.
> >
>

Re: multi-threaded compiling

[Mischa Baars]

Sorry. I mean:

for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ )); do
wait -n; echo $?; done;

doesn't function. With an ampersand instead of a semicolon. Why does it
function when called from a script and why does it fail when called from
the command line?

On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars 
wrote:

> You mean:
>
> for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;
>
> with one command and one wait in a single loop. And this does execute on
> the command line. How interesting!
>
> for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
> wait -n; echo $?; done;
>
> Because this doesn't and to be honest, I needed the pid and its index to
> retrieve gcc's output from a log file array afterwards.
>
> On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
> wrote:
>
>>
>>
>> On Mon, Mar 11, 2024, 19:22 Mischa Baars 
>> wrote:
>>
>>> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
>>> wrote:
>>>
 i also completly dont get ur issue

 f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=

   while [[ -v f[++i] ]] ; do
  (( ++r > threads )) &&
 wait -n
 gcc -c "${f[i]}" &
   done

>>>
>>> How nice!
>>>
>>> wait -n exit 1 & echo $?
>>>
>>
>> doesnt need a pid
>> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>>
>> You got me the solution :) Except that wait expects a pid after -n.
>>>
>>> Maybe
>>>
>>> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>>>
>>> is what you meant? The equivalence of sequential execution?
>>>
>>> First think, then do magic.
>>>
>>>


>>>

 On Mon, Mar 11, 2024, 18:16 Mischa Baars 
 wrote:

> Hello Paul,
>
> It seems I'm awake a little longer than you are.
>
> The second paragraph as you see it, belongs to 1)
> The fourth paragraph as you see it, belongs to 2)
>
> The actual command invocations (a Makefile, a make.sh script) can be
> found
> in the attachment, as indicated on the first line of the mail. In the
> attachment there are two directories, one and two, belonging to 1) and
> 2)
> respectively.
>
> I'm not into Vulcan mindmelds, so I hope everything from the first mail
> makes sense to you and everyone on this mailing list now.
>
> Best regards,
> Mischa Baars.
>
> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>
> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
> > > I've attached the problem. It consists of two parts:
> > >
> > > 1) multi-threaded bash script and / or multi-threaded Makefile
> > >
> > > Running bash script functions as expected, but executing the same
> > > line of code with make and / or the command line, does not
> function.
> > > Perhaps someone could explain to me why?
> > >
> > > 2) passing a string argument from a bash script and / or Makefile
> to
> > > the gcc -D option
> > >
> > > Running the makefile functions as expected, but I have not been
> able
> > > to get similar code to work from a bash script. Can someone please
> > > explain to me what I'm doing wrong?
> >
> > I don't understand the problem.  In the third paragraph above you say
> > the bash script works as expected and the makefile doesn't work, but
> in
> > the last paragraph you say that the makefile works as expected but
> you
> > can't get it to work in bash.
> >
> > Please provide actual command invocations (cut and pasted) showing
> the
> > output you received and explaining exactly what is wrong with it.
> >
> > But before you do that, be aware that make does NOT invoke /bin/bash
> as
> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is actually
> an
> > alias for bash.  On other systems it isn't.
> >
> > If you want your makefile to always use bash as its shell, you should
> > add an explicit:
> >
> > SHELL := /bin/bash
> >
> > to your makefile to force it.  Maybe that will solve your problem.
> If
> > not we'll need details such as I mention above.
> >
>

Re: multi-threaded compiling

[alex xmb sw ratchev]

the logic between my code

1 threads_max
2 loop
3 inside loop , do if run is > than threads_max then wait -n one
then 4 spawn thread

i dont get ur points

On Mon, Mar 11, 2024, 19:55 Mischa Baars 
wrote:

> Sorry. I mean:
>
> for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ )); do
> wait -n; echo $?; done;
>
> doesn't function. With an ampersand instead of a semicolon. Why does it
> function when called from a script and why does it fail when called from
> the command line?
>
> On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars 
> wrote:
>
>> You mean:
>>
>> for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;
>>
>> with one command and one wait in a single loop. And this does execute on
>> the command line. How interesting!
>>
>> for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
>> wait -n; echo $?; done;
>>
>> Because this doesn't and to be honest, I needed the pid and its index to
>> retrieve gcc's output from a log file array afterwards.
>>
>> On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
>> wrote:
>>
>>>
>>>
>>> On Mon, Mar 11, 2024, 19:22 Mischa Baars 
>>> wrote:
>>>
 On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
 wrote:

> i also completly dont get ur issue
>
> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>
>   while [[ -v f[++i] ]] ; do
>  (( ++r > threads )) &&
> wait -n
> gcc -c "${f[i]}" &
>   done
>

 How nice!

 wait -n exit 1 & echo $?

>>>
>>> doesnt need a pid
>>> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>>>
>>> You got me the solution :) Except that wait expects a pid after -n.

 Maybe

 for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;

 is what you meant? The equivalence of sequential execution?

 First think, then do magic.


>
>

>
> On Mon, Mar 11, 2024, 18:16 Mischa Baars 
> wrote:
>
>> Hello Paul,
>>
>> It seems I'm awake a little longer than you are.
>>
>> The second paragraph as you see it, belongs to 1)
>> The fourth paragraph as you see it, belongs to 2)
>>
>> The actual command invocations (a Makefile, a make.sh script) can be
>> found
>> in the attachment, as indicated on the first line of the mail. In the
>> attachment there are two directories, one and two, belonging to 1)
>> and 2)
>> respectively.
>>
>> I'm not into Vulcan mindmelds, so I hope everything from the first
>> mail
>> makes sense to you and everyone on this mailing list now.
>>
>> Best regards,
>> Mischa Baars.
>>
>> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>>
>> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
>> > > I've attached the problem. It consists of two parts:
>> > >
>> > > 1) multi-threaded bash script and / or multi-threaded Makefile
>> > >
>> > > Running bash script functions as expected, but executing the same
>> > > line of code with make and / or the command line, does not
>> function.
>> > > Perhaps someone could explain to me why?
>> > >
>> > > 2) passing a string argument from a bash script and / or Makefile
>> to
>> > > the gcc -D option
>> > >
>> > > Running the makefile functions as expected, but I have not been
>> able
>> > > to get similar code to work from a bash script. Can someone please
>> > > explain to me what I'm doing wrong?
>> >
>> > I don't understand the problem.  In the third paragraph above you
>> say
>> > the bash script works as expected and the makefile doesn't work,
>> but in
>> > the last paragraph you say that the makefile works as expected but
>> you
>> > can't get it to work in bash.
>> >
>> > Please provide actual command invocations (cut and pasted) showing
>> the
>> > output you received and explaining exactly what is wrong with it.
>> >
>> > But before you do that, be aware that make does NOT invoke
>> /bin/bash as
>> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is
>> actually an
>> > alias for bash.  On other systems it isn't.
>> >
>> > If you want your makefile to always use bash as its shell, you
>> should
>> > add an explicit:
>> >
>> > SHELL := /bin/bash
>> >
>> > to your makefile to force it.  Maybe that will solve your problem.
>> If
>> > not we'll need details such as I mention above.
>> >
>>
>

Re: multi-threaded compiling

[alex xmb sw ratchev]

On Mon, Mar 11, 2024, 20:03 alex xmb sw ratchev  wrote:

> the logic between my code
>
> 1 threads_max
> 2 loop
> 3 inside loop , do if run is > than threads_max then wait -n one
> then 4 spawn thread
>

3 if run isnt more than max , simply ignore and spawn thread in next cmd

i dont get ur points
>
> On Mon, Mar 11, 2024, 19:55 Mischa Baars 
> wrote:
>
>> Sorry. I mean:
>>
>> for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ )); do
>> wait -n; echo $?; done;
>>
>> doesn't function. With an ampersand instead of a semicolon. Why does it
>> function when called from a script and why does it fail when called from
>> the command line?
>>
>> On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars <
>> mjbaars1977.bac...@gmail.com> wrote:
>>
>>> You mean:
>>>
>>> for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;
>>>
>>> with one command and one wait in a single loop. And this does execute on
>>> the command line. How interesting!
>>>
>>> for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
>>> wait -n; echo $?; done;
>>>
>>> Because this doesn't and to be honest, I needed the pid and its index to
>>> retrieve gcc's output from a log file array afterwards.
>>>
>>> On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
>>> wrote:
>>>


 On Mon, Mar 11, 2024, 19:22 Mischa Baars 
 wrote:

> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev 
> wrote:
>
>> i also completly dont get ur issue
>>
>> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>>
>>   while [[ -v f[++i] ]] ; do
>>  (( ++r > threads )) &&
>> wait -n
>> gcc -c "${f[i]}" &
>>   done
>>
>
> How nice!
>
> wait -n exit 1 & echo $?
>

 doesnt need a pid
 1 : 1 as i wrote it , excepts add 'wait' as new last line

 You got me the solution :) Except that wait expects a pid after -n.
>
> Maybe
>
> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>
> is what you meant? The equivalence of sequential execution?
>
> First think, then do magic.
>
>
>>
>>
>
>>
>> On Mon, Mar 11, 2024, 18:16 Mischa Baars <
>> mjbaars1977.bac...@gmail.com> wrote:
>>
>>> Hello Paul,
>>>
>>> It seems I'm awake a little longer than you are.
>>>
>>> The second paragraph as you see it, belongs to 1)
>>> The fourth paragraph as you see it, belongs to 2)
>>>
>>> The actual command invocations (a Makefile, a make.sh script) can be
>>> found
>>> in the attachment, as indicated on the first line of the mail. In the
>>> attachment there are two directories, one and two, belonging to 1)
>>> and 2)
>>> respectively.
>>>
>>> I'm not into Vulcan mindmelds, so I hope everything from the first
>>> mail
>>> makes sense to you and everyone on this mailing list now.
>>>
>>> Best regards,
>>> Mischa Baars.
>>>
>>> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>>>
>>> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
>>> > > I've attached the problem. It consists of two parts:
>>> > >
>>> > > 1) multi-threaded bash script and / or multi-threaded Makefile
>>> > >
>>> > > Running bash script functions as expected, but executing the same
>>> > > line of code with make and / or the command line, does not
>>> function.
>>> > > Perhaps someone could explain to me why?
>>> > >
>>> > > 2) passing a string argument from a bash script and / or
>>> Makefile to
>>> > > the gcc -D option
>>> > >
>>> > > Running the makefile functions as expected, but I have not been
>>> able
>>> > > to get similar code to work from a bash script. Can someone
>>> please
>>> > > explain to me what I'm doing wrong?
>>> >
>>> > I don't understand the problem.  In the third paragraph above you
>>> say
>>> > the bash script works as expected and the makefile doesn't work,
>>> but in
>>> > the last paragraph you say that the makefile works as expected but
>>> you
>>> > can't get it to work in bash.
>>> >
>>> > Please provide actual command invocations (cut and pasted) showing
>>> the
>>> > output you received and explaining exactly what is wrong with it.
>>> >
>>> > But before you do that, be aware that make does NOT invoke
>>> /bin/bash as
>>> > its shell.  It invokes /bin/sh.  On some systems /bin/sh is
>>> actually an
>>> > alias for bash.  On other systems it isn't.
>>> >
>>> > If you want your makefile to always use bash as its shell, you
>>> should
>>> > add an explicit:
>>> >
>>> > SHELL := /bin/bash
>>> >
>>> > to your makefile to force it.  Maybe that will solve your
>>> problem.  If
>>> > not we'll need details such as I mention above.
>>> >
>>>
>>

Re: multi-threaded compiling

[Mischa Baars]

That's not really an answer to the question.

Also I don't think that gives you an exit status for each 'exit $i'
started. I need that exit status.


On Mon, 11 Mar 2024, 20:03 alex xmb sw ratchev,  wrote:

>
>
> On Mon, Mar 11, 2024, 20:03 alex xmb sw ratchev  wrote:
>
>> the logic between my code
>>
>> 1 threads_max
>> 2 loop
>> 3 inside loop , do if run is > than threads_max then wait -n one
>> then 4 spawn thread
>>
>
> 3 if run isnt more than max , simply ignore and spawn thread in next cmd
>
> i dont get ur points
>>
>> On Mon, Mar 11, 2024, 19:55 Mischa Baars 
>> wrote:
>>
>>> Sorry. I mean:
>>>
>>> for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ ));
>>> do wait -n; echo $?; done;
>>>
>>> doesn't function. With an ampersand instead of a semicolon. Why does it
>>> function when called from a script and why does it fail when called from
>>> the command line?
>>>
>>> On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars <
>>> mjbaars1977.bac...@gmail.com> wrote:
>>>
 You mean:

 for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;

 with one command and one wait in a single loop. And this does execute
 on the command line. How interesting!

 for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ ));
 do wait -n; echo $?; done;

 Because this doesn't and to be honest, I needed the pid and its index
 to retrieve gcc's output from a log file array afterwards.

 On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
 wrote:

>
>
> On Mon, Mar 11, 2024, 19:22 Mischa Baars 
> wrote:
>
>> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev <
>> fxmb...@gmail.com> wrote:
>>
>>> i also completly dont get ur issue
>>>
>>> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=
>>>
>>>   while [[ -v f[++i] ]] ; do
>>>  (( ++r > threads )) &&
>>> wait -n
>>> gcc -c "${f[i]}" &
>>>   done
>>>
>>
>> How nice!
>>
>> wait -n exit 1 & echo $?
>>
>
> doesnt need a pid
> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>
> You got me the solution :) Except that wait expects a pid after -n.
>>
>> Maybe
>>
>> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>>
>> is what you meant? The equivalence of sequential execution?
>>
>> First think, then do magic.
>>
>>
>>>
>>>
>>
>>>
>>> On Mon, Mar 11, 2024, 18:16 Mischa Baars <
>>> mjbaars1977.bac...@gmail.com> wrote:
>>>
 Hello Paul,

 It seems I'm awake a little longer than you are.

 The second paragraph as you see it, belongs to 1)
 The fourth paragraph as you see it, belongs to 2)

 The actual command invocations (a Makefile, a make.sh script) can
 be found
 in the attachment, as indicated on the first line of the mail. In
 the
 attachment there are two directories, one and two, belonging to 1)
 and 2)
 respectively.

 I'm not into Vulcan mindmelds, so I hope everything from the first
 mail
 makes sense to you and everyone on this mailing list now.

 Best regards,
 Mischa Baars.

 On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:

 > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
 > > I've attached the problem. It consists of two parts:
 > >
 > > 1) multi-threaded bash script and / or multi-threaded Makefile
 > >
 > > Running bash script functions as expected, but executing the
 same
 > > line of code with make and / or the command line, does not
 function.
 > > Perhaps someone could explain to me why?
 > >
 > > 2) passing a string argument from a bash script and / or
 Makefile to
 > > the gcc -D option
 > >
 > > Running the makefile functions as expected, but I have not been
 able
 > > to get similar code to work from a bash script. Can someone
 please
 > > explain to me what I'm doing wrong?
 >
 > I don't understand the problem.  In the third paragraph above you
 say
 > the bash script works as expected and the makefile doesn't work,
 but in
 > the last paragraph you say that the makefile works as expected
 but you
 > can't get it to work in bash.
 >
 > Please provide actual command invocations (cut and pasted)
 showing the
 > output you received and explaining exactly what is wrong with it.
 >
 > But before you do that, be aware that make does NOT invoke
 /bin/bash as
 > its shell.  It invokes /bin/sh.  On some systems /bin/sh is
 actually an
 > alias for bash.  On other systems it isn't.
>>

Re: multi-threaded compiling

[Chet Ramey]

On 3/11/24 2:50 PM, Mischa Baars wrote:

Which sort of brings us back to the original question I suppose. Who does
that line of code function from a script and why does it fail from the
command line? 


Job control and when the shell notifies the user about job completion,
most likely, two of the relevant things that differ between interactive
and non-interactive shells.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: multi-threaded compiling

[alex xmb sw ratchev]

afik wait will exit with that $? of returned

On Mon, Mar 11, 2024, 20:13 Mischa Baars 
wrote:

> That's not really an answer to the question.
>
> Also I don't think that gives you an exit status for each 'exit $i'
> started. I need that exit status.
>
>
> On Mon, 11 Mar 2024, 20:03 alex xmb sw ratchev,  wrote:
>
>>
>>
>> On Mon, Mar 11, 2024, 20:03 alex xmb sw ratchev 
>> wrote:
>>
>>> the logic between my code
>>>
>>> 1 threads_max
>>> 2 loop
>>> 3 inside loop , do if run is > than threads_max then wait -n one
>>> then 4 spawn thread
>>>
>>
>> 3 if run isnt more than max , simply ignore and spawn thread in next cmd
>>
>> i dont get ur points
>>>
>>> On Mon, Mar 11, 2024, 19:55 Mischa Baars 
>>> wrote:
>>>
 Sorry. I mean:

 for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ ));
 do wait -n; echo $?; done;

 doesn't function. With an ampersand instead of a semicolon. Why does it
 function when called from a script and why does it fail when called from
 the command line?

 On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars <
 mjbaars1977.bac...@gmail.com> wrote:

> You mean:
>
> for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done;
>
> with one command and one wait in a single loop. And this does execute
> on the command line. How interesting!
>
> for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ ));
> do wait -n; echo $?; done;
>
> Because this doesn't and to be honest, I needed the pid and its index
> to retrieve gcc's output from a log file array afterwards.
>
> On Mon, Mar 11, 2024 at 7:25 PM alex xmb sw ratchev 
> wrote:
>
>>
>>
>> On Mon, Mar 11, 2024, 19:22 Mischa Baars <
>> mjbaars1977.bac...@gmail.com> wrote:
>>
>>> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev <
>>> fxmb...@gmail.com> wrote:
>>>
 i also completly dont get ur issue

 f=( a.c b.c .. ) threads=$( nproc ) i=-1 r=

   while [[ -v f[++i] ]] ; do
  (( ++r > threads )) &&
 wait -n
 gcc -c "${f[i]}" &
   done

>>>
>>> How nice!
>>>
>>> wait -n exit 1 & echo $?
>>>
>>
>> doesnt need a pid
>> 1 : 1 as i wrote it , excepts add 'wait' as new last line
>>
>> You got me the solution :) Except that wait expects a pid after -n.
>>>
>>> Maybe
>>>
>>> for (( i=0; i<32; i++ )); do exit 1 & wait -n $!; echo $?; done;
>>>
>>> is what you meant? The equivalence of sequential execution?
>>>
>>> First think, then do magic.
>>>
>>>


>>>

 On Mon, Mar 11, 2024, 18:16 Mischa Baars <
 mjbaars1977.bac...@gmail.com> wrote:

> Hello Paul,
>
> It seems I'm awake a little longer than you are.
>
> The second paragraph as you see it, belongs to 1)
> The fourth paragraph as you see it, belongs to 2)
>
> The actual command invocations (a Makefile, a make.sh script) can
> be found
> in the attachment, as indicated on the first line of the mail. In
> the
> attachment there are two directories, one and two, belonging to 1)
> and 2)
> respectively.
>
> I'm not into Vulcan mindmelds, so I hope everything from the first
> mail
> makes sense to you and everyone on this mailing list now.
>
> Best regards,
> Mischa Baars.
>
> On Mon, Mar 11, 2024 at 5:01 PM Paul Smith  wrote:
>
> > On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote:
> > > I've attached the problem. It consists of two parts:
> > >
> > > 1) multi-threaded bash script and / or multi-threaded Makefile
> > >
> > > Running bash script functions as expected, but executing the
> same
> > > line of code with make and / or the command line, does not
> function.
> > > Perhaps someone could explain to me why?
> > >
> > > 2) passing a string argument from a bash script and / or
> Makefile to
> > > the gcc -D option
> > >
> > > Running the makefile functions as expected, but I have not
> been able
> > > to get similar code to work from a bash script. Can someone
> please
> > > explain to me what I'm doing wrong?
> >
> > I don't understand the problem.  In the third paragraph above
> you say
> > the bash script works as expected and the makefile doesn't work,
> but in
> > the last paragraph you say that the makefile works as expected
> but you
> > can't get it to work in bash.
> >
> > Please provide actual command invocations (cut and pasted)
> showing the
> > output you received and explaini

Re: multi-threaded compiling

[Greg Wooledge]

> On Mon, Mar 11, 2024, 20:13 Mischa Baars 
> wrote:
> 
> > Also I don't think that gives you an exit status for each 'exit $i'
> > started. I need that exit status.

"wait -n" without a PID won't help you, then.  You don't get the PID or
job ID that terminated, and you don't get the exit status.  It's only
of interest if you're trying to do something like "run these 100 jobs,
5 at a time" without storing their exit statuses.

If you need to reap each individual job and store its exit status indexed
by its PID, then you're probably going to need something like:


#!/bin/bash
i=0 pid=() status=()
for job in ...; do
longrunner "$job" & pid[i++]=$!
done

for ((i=0; i < ${#pid[@]}; i++)); do
wait "${pid[i]}"; status[i]=$#
done


You won't be able to take advantage of "wait -n"'s ability to react
to the first job that finishes.  You'll end up reaping each job in the
order they started, not the order they finished.

Re: multi-threaded compiling

[Paul Smith]

On Mon, 2024-03-11 at 15:36 -0400, Greg Wooledge wrote:
> You won't be able to take advantage of "wait -n"'s ability to react
> to the first job that finishes.  You'll end up reaping each job in
> the order they started, not the order they finished.

It feels to me like you're trying to reproduce make's already-available
parallel build support, in a kind of kludgy way with bash.

Maybe this is an XY problem and you should step back and describe WHY
you're trying to create this behavior using bash instead of just using
make's support for parallel jobs.

What about make's parallel jobs doesn't meet your requirements (or put
another way, what actually are your requirements)?

Re: multi-threaded compiling

[Mischa Baars]

On Mon, 11 Mar 2024, 20:20 Chet Ramey,  wrote:

> On 3/11/24 2:50 PM, Mischa Baars wrote:
> > Which sort of brings us back to the original question I suppose. Who does
> > that line of code function from a script and why does it fail from the
> > command line?
>
> Job control and when the shell notifies the user about job completion,
> most likely, two of the relevant things that differ between interactive
> and non-interactive shells.
>

Wasn't expecting any better answer, but thanks anyway.

And the script from directory 'two'? How do I pass this string through the
CFLAGS variable to the compiler? What am I doing wrong in #3 of the script?


> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
>

Re: multi-threaded compiling

[Chet Ramey]

On 3/11/24 2:46 PM, Mischa Baars wrote:

You mean:



for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
wait -n; echo $?; done;

Because this doesn't and to be honest, I needed the pid and its index to
retrieve gcc's output from a log file array afterwards.


If you want the pid, take a look at the -p option to wait.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: multi-threaded compiling

[Mischa Baars]

On Mon, 11 Mar 2024, 20:36 Greg Wooledge,  wrote:

> > On Mon, Mar 11, 2024, 20:13 Mischa Baars 
> > wrote:
> >
> > > Also I don't think that gives you an exit status for each 'exit $i'
> > > started. I need that exit status.
>
> "wait -n" without a PID won't help you, then.  You don't get the PID or
> job ID that terminated, and you don't get the exit status.  It's only
> of interest if you're trying to do something like "run these 100 jobs,
> 5 at a time" without storing their exit statuses.
>
> If you need to reap each individual job and store its exit status indexed
> by its PID, then you're probably going to need something like:
>
>
> #!/bin/bash
> i=0 pid=() status=()
> for job in ...; do
> longrunner "$job" & pid[i++]=$!
> done
>
> for ((i=0; i < ${#pid[@]}; i++)); do
> wait "${pid[i]}"; status[i]=$#
> done
>
>
> You won't be able to take advantage of "wait -n"'s ability to react
> to the first job that finishes.  You'll end up reaping each job in the
> order they started, not the order they finished.
>

Exactly my thought, but does that really matter and do we even have another
option?

The glibc wait(2) returns a pid to that purpose. This bash wait doesn't.

>

Re: multi-threaded compiling

[alex xmb sw ratchev]

bash s wait returns the exit code of the proc from the job

On Mon, Mar 11, 2024, 20:52 Mischa Baars 
wrote:

> On Mon, 11 Mar 2024, 20:36 Greg Wooledge,  wrote:
>
> > > On Mon, Mar 11, 2024, 20:13 Mischa Baars  >
> > > wrote:
> > >
> > > > Also I don't think that gives you an exit status for each 'exit $i'
> > > > started. I need that exit status.
> >
> > "wait -n" without a PID won't help you, then.  You don't get the PID or
> > job ID that terminated, and you don't get the exit status.  It's only
> > of interest if you're trying to do something like "run these 100 jobs,
> > 5 at a time" without storing their exit statuses.
> >
> > If you need to reap each individual job and store its exit status indexed
> > by its PID, then you're probably going to need something like:
> >
> >
> > #!/bin/bash
> > i=0 pid=() status=()
> > for job in ...; do
> > longrunner "$job" & pid[i++]=$!
> > done
> >
> > for ((i=0; i < ${#pid[@]}; i++)); do
> > wait "${pid[i]}"; status[i]=$#
> > done
> >
> >
> > You won't be able to take advantage of "wait -n"'s ability to react
> > to the first job that finishes.  You'll end up reaping each job in the
> > order they started, not the order they finished.
> >
>
> Exactly my thought, but does that really matter and do we even have another
> option?
>
> The glibc wait(2) returns a pid to that purpose. This bash wait doesn't.
>
> >
>

Re: multi-threaded compiling

[Kerin Millar]

On Mon, 11 Mar 2024 15:36:48 -0400
Greg Wooledge  wrote:

> > On Mon, Mar 11, 2024, 20:13 Mischa Baars 
> > wrote:
> > 
> > > Also I don't think that gives you an exit status for each 'exit $i'
> > > started. I need that exit status.
> 
> "wait -n" without a PID won't help you, then.  You don't get the PID or
> job ID that terminated, and you don't get the exit status.  It's only

It does convey the exit status.

> of interest if you're trying to do something like "run these 100 jobs,
> 5 at a time" without storing their exit statuses.

The pid can be obtained with the -p option, as of 5.1. Below is a synthetic 
example of how it might be put into practice.

#!/bin/bash

declare -A job_by status_by
max_jobs=4
jobs=0

wait_next() {
local pid
wait -n -p pid
status_by[$pid]=$?
unset -v 'job_by[$pid]'
}

worker() {
sleep "$(( RANDOM % 5 ))"
exit "$(( RANDOM % 2 ))"
}

for (( i = 0; i < 16; ++i )); do
(( jobs++ < max_jobs )) || wait_next
worker & job_by[$!]=
done

while (( ${#job_by[@]} )); do
wait_next
done

declare -p status_by

-- 
Kerin Millar

Re: multi-threaded compiling

[Mischa Baars]

Like this:

seconds=1; for (( i=0;i<32;i++ )); do exit ${i} & done; sleep ${seconds};
for (( i=0;i<32;i++ )); do wait -np pid; e=${?}; echo "$(printf %3u ${i})
pid ${pid} exit ${e}"; done;

which again only works when called from a script. I have been looking for
the -p option. Guess I overlooked it.

Now you would have to look up the pid in the pid array of started 'exit
${i}''s to lookup the log file corresponding to that index. I suppose
waiting for the processes to end in the same order as they were started
isn't that bad a solution. You then have that index right at hand.

On Mon, Mar 11, 2024 at 8:49 PM Chet Ramey  wrote:

> On 3/11/24 2:46 PM, Mischa Baars wrote:
> > You mean:
>
> > for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do
> > wait -n; echo $?; done;
> >
> > Because this doesn't and to be honest, I needed the pid and its index to
> > retrieve gcc's output from a log file array afterwards.
>
> If you want the pid, take a look at the -p option to wait.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
>

Re: multi-threaded compiling

[Mischa Baars]

On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:

> On Mon, 11 Mar 2024 15:36:48 -0400
> Greg Wooledge  wrote:
>
> > > On Mon, Mar 11, 2024, 20:13 Mischa Baars  >
> > > wrote:
> > >
> > > > Also I don't think that gives you an exit status for each 'exit $i'
> > > > started. I need that exit status.
> >
> > "wait -n" without a PID won't help you, then.  You don't get the PID or
> > job ID that terminated, and you don't get the exit status.  It's only
>
> It does convey the exit status.
>
> > of interest if you're trying to do something like "run these 100 jobs,
> > 5 at a time" without storing their exit statuses.
>
> The pid can be obtained with the -p option, as of 5.1. Below is a
> synthetic example of how it might be put into practice.
>
> #!/bin/bash
>
> declare -A job_by status_by
> max_jobs=4
> jobs=0
>
> wait_next() {
> local pid
> wait -n -p pid
> status_by[$pid]=$?
>

How exactly is this indexing implemented internally? Does a first number on
index n take m bits (using a linked list) or does it take n * m bits (using
realloc(3))?

unset -v 'job_by[$pid]'
> }
>
> worker() {
> sleep "$(( RANDOM % 5 ))"
> exit "$(( RANDOM % 2 ))"
> }
>
> for (( i = 0; i < 16; ++i )); do
> (( jobs++ < max_jobs )) || wait_next
> worker & job_by[$!]=
> done
>
> while (( ${#job_by[@]} )); do
> wait_next
> done
>
> declare -p status_by
>
> --
> Kerin Millar
>
>

Re: multi-threaded compiling

[Greg Wooledge]

On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote:
> On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:
> > The pid can be obtained with the -p option, as of 5.1. Below is a
> > synthetic example of how it might be put into practice.

I'd forgotten about that one.  A recent addition, and one I've never used
yet.

> > #!/bin/bash
> >
> > declare -A job_by status_by
> > max_jobs=4
> > jobs=0
> >
> > wait_next() {
> > local pid
> > wait -n -p pid
> > status_by[$pid]=$?
> >
> 
> How exactly is this indexing implemented internally? Does a first number on
> index n take m bits (using a linked list) or does it take n * m bits (using
> realloc(3))?

"declare -A" makes it an associative array (hash table).

Without that declare -A, it would be a sparsely indexed array.

In either case, it doesn't just blindly allocate millions of bytes of
memory.  It uses something slimmer.

Re: multi-threaded compiling

[Chet Ramey]

On 3/11/24 3:44 PM, Mischa Baars wrote:
On Mon, 11 Mar 2024, 20:20 Chet Ramey, > wrote:


On 3/11/24 2:50 PM, Mischa Baars wrote:
 > Which sort of brings us back to the original question I suppose. Who
does
 > that line of code function from a script and why does it fail from the
 > command line?

Job control and when the shell notifies the user about job completion,
most likely, two of the relevant things that differ between interactive
and non-interactive shells.


Wasn't expecting any better answer, but thanks anyway.


OK, here's the longer answer. When the shell is interactive, and job
control is enabled, the shell prints job completion notifications to
stdout at command boundaries. Once the shell notifies the user of a
job's completion, it is no longer `new' (or the `next', whichever
mnemonic you prefer), and `wait -n' will not return it.

So if you include the complete output of an interactive shell executing
the above list (with the obvious corrections already discussed), you'll
get a series of

32 messages about job creation
32 notifications of job exits, with exit status (they exit immediately,
after all)
32 error messages because the user has already been notified of the
job's completion and the shell has reaped it (so it is no longer the
`next' job to exit)

When the shell is not interactive, it only prints job notifications to
stdout when the job is terminated by a signal, so the jobs are available
to `wait -n'.

There was quite a long discussion about this aspect of `wait -n' behavior
a couple of months ago, starting at

https://lists.gnu.org/archive/html/bug-bash/2024-01/msg00104.html


And the script from directory 'two'? How do I pass this string through the 
CFLAGS variable to the compiler? What am I doing wrong in #3 of the script?


You're not taking word splitting into account. It seems like you want
selective word splitting -- for the embedded double quotes in the various
elements of CFLAGS to have some semantic meaning and somehow inhibit
word splitting. Quotes are special to the parser, not when they are in
the results of word expansion. If you want the parser to evaluate the
results of word expansion, you're going to have to run it through the
parser again, using something like `eval'.

It's clearer if you replace `gcc' in your command with

printf '<%s> '

and `./main' with `echo'.


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: multi-threaded compiling

[alex xmb sw ratchev]

~ $ bash xmb.smallt
pid 14333 cmd t2 returned 3
pid 14332 cmd sleep 1 returned 0

~ $ cat xmb.smallt
#!/bin/bash

 run() {
local IFS=' ' run=$*
eval "$run &"
me[$!]=$run
 }

 wa() {
local pid
wait -n -p pid
printf %s\\n "pid $pid cmd ${me[pid]} returned $?"
 }

 t2() {
sleep .75
return 3
 }

run sleep 1
run t2

wa
wa

On Mon, Mar 11, 2024, 22:24 Greg Wooledge  wrote:

> On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote:
> > On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:
> > > The pid can be obtained with the -p option, as of 5.1. Below is a
> > > synthetic example of how it might be put into practice.
>
> I'd forgotten about that one.  A recent addition, and one I've never used
> yet.
>
> > > #!/bin/bash
> > >
> > > declare -A job_by status_by
> > > max_jobs=4
> > > jobs=0
> > >
> > > wait_next() {
> > > local pid
> > > wait -n -p pid
> > > status_by[$pid]=$?
> > >
> >
> > How exactly is this indexing implemented internally? Does a first number
> on
> > index n take m bits (using a linked list) or does it take n * m bits
> (using
> > realloc(3))?
>
> "declare -A" makes it an associative array (hash table).
>
> Without that declare -A, it would be a sparsely indexed array.
>
> In either case, it doesn't just blindly allocate millions of bytes of
> memory.  It uses something slimmer.
>
>


xmb.smallt
Description: Binary data

Re: multi-threaded compiling

[alex xmb sw ratchev]

On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev  wrote:

> ~ $ bash xmb.smallt
> pid 14333 cmd t2 returned 3
> pid 14332 cmd sleep 1 returned 0
>
> ~ $ cat xmb.smallt
> #!/bin/bash
>
>  run() {
> local IFS=' ' run=$*
> eval "$run &"
> me[$!]=$run
>  }
>

alternativly to mention aliases usage

shopt -s expand_aliases

then in func
IFS=' '
alias torun=$*

then
torun &

 wa() {
> local pid
> wait -n -p pid
> printf %s\\n "pid $pid cmd ${me[pid]} returned $?"
>  }
>
>  t2() {
> sleep .75
> return 3
>  }
>
> run sleep 1
> run t2
>
> wa
> wa
>
> On Mon, Mar 11, 2024, 22:24 Greg Wooledge  wrote:
>
>> On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote:
>> > On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:
>> > > The pid can be obtained with the -p option, as of 5.1. Below is a
>> > > synthetic example of how it might be put into practice.
>>
>> I'd forgotten about that one.  A recent addition, and one I've never used
>> yet.
>>
>> > > #!/bin/bash
>> > >
>> > > declare -A job_by status_by
>> > > max_jobs=4
>> > > jobs=0
>> > >
>> > > wait_next() {
>> > > local pid
>> > > wait -n -p pid
>> > > status_by[$pid]=$?
>> > >
>> >
>> > How exactly is this indexing implemented internally? Does a first
>> number on
>> > index n take m bits (using a linked list) or does it take n * m bits
>> (using
>> > realloc(3))?
>>
>> "declare -A" makes it an associative array (hash table).
>>
>> Without that declare -A, it would be a sparsely indexed array.
>>
>> In either case, it doesn't just blindly allocate millions of bytes of
>> memory.  It uses something slimmer.
>>
>>

Re: multi-threaded compiling

[alex xmb sw ratchev]

On Mon, Mar 11, 2024, 22:40 alex xmb sw ratchev  wrote:

>
>
> On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev  wrote:
>
>> ~ $ bash xmb.smallt
>> pid 14333 cmd t2 returned 3
>> pid 14332 cmd sleep 1 returned 0
>>
>> ~ $ cat xmb.smallt
>> #!/bin/bash
>>
>>  run() {
>> local IFS=' ' run=$*
>> eval "$run &"
>> me[$!]=$run
>>  }
>>
>
> alternativly to mention aliases usage
>
> shopt -s expand_aliases
>
> then in func
> IFS=' '
> alias torun=$*
>
> then
> torun &
>

still needs then one more $* expand to set cmds[$!] to it

 wa() {
>> local pid
>> wait -n -p pid
>> printf %s\\n "pid $pid cmd ${me[pid]} returned $?"
>>  }
>>
>>  t2() {
>> sleep .75
>> return 3
>>  }
>>
>> run sleep 1
>> run t2
>>
>> wa
>> wa
>>
>> On Mon, Mar 11, 2024, 22:24 Greg Wooledge  wrote:
>>
>>> On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote:
>>> > On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:
>>> > > The pid can be obtained with the -p option, as of 5.1. Below is a
>>> > > synthetic example of how it might be put into practice.
>>>
>>> I'd forgotten about that one.  A recent addition, and one I've never used
>>> yet.
>>>
>>> > > #!/bin/bash
>>> > >
>>> > > declare -A job_by status_by
>>> > > max_jobs=4
>>> > > jobs=0
>>> > >
>>> > > wait_next() {
>>> > > local pid
>>> > > wait -n -p pid
>>> > > status_by[$pid]=$?
>>> > >
>>> >
>>> > How exactly is this indexing implemented internally? Does a first
>>> number on
>>> > index n take m bits (using a linked list) or does it take n * m bits
>>> (using
>>> > realloc(3))?
>>>
>>> "declare -A" makes it an associative array (hash table).
>>>
>>> Without that declare -A, it would be a sparsely indexed array.
>>>
>>> In either case, it doesn't just blindly allocate millions of bytes of
>>> memory.  It uses something slimmer.
>>>
>>>

Re: multi-threaded compiling

[alex xmb sw ratchev]

On Mon, Mar 11, 2024, 22:40 alex xmb sw ratchev  wrote:

>
>
> On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev  wrote:
>
>> ~ $ bash xmb.smallt
>> pid 14333 cmd t2 returned 3
>> pid 14332 cmd sleep 1 returned 0
>>
>> ~ $ cat xmb.smallt
>> #!/bin/bash
>>
>>  run() {
>> local IFS=' ' run=$*
>> eval "$run &"
>> me[$!]=$run
>>  }
>>
>
> alternativly to mention aliases usage
>
> shopt -s expand_aliases
>
> then in func
> IFS=' '
> alias torun=$*
>

and with quotes prolly ye =/

then
> torun &
>
>  wa() {
>> local pid
>> wait -n -p pid
>> printf %s\\n "pid $pid cmd ${me[pid]} returned $?"
>>  }
>>
>>  t2() {
>> sleep .75
>> return 3
>>  }
>>
>> run sleep 1
>> run t2
>>
>> wa
>> wa
>>
>> On Mon, Mar 11, 2024, 22:24 Greg Wooledge  wrote:
>>
>>> On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote:
>>> > On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:
>>> > > The pid can be obtained with the -p option, as of 5.1. Below is a
>>> > > synthetic example of how it might be put into practice.
>>>
>>> I'd forgotten about that one.  A recent addition, and one I've never used
>>> yet.
>>>
>>> > > #!/bin/bash
>>> > >
>>> > > declare -A job_by status_by
>>> > > max_jobs=4
>>> > > jobs=0
>>> > >
>>> > > wait_next() {
>>> > > local pid
>>> > > wait -n -p pid
>>> > > status_by[$pid]=$?
>>> > >
>>> >
>>> > How exactly is this indexing implemented internally? Does a first
>>> number on
>>> > index n take m bits (using a linked list) or does it take n * m bits
>>> (using
>>> > realloc(3))?
>>>
>>> "declare -A" makes it an associative array (hash table).
>>>
>>> Without that declare -A, it would be a sparsely indexed array.
>>>
>>> In either case, it doesn't just blindly allocate millions of bytes of
>>> memory.  It uses something slimmer.
>>>
>>>

Re: multi-threaded compiling

[alex xmb sw ratchev]

a version with success or fail handling

~ $ bash xmb.smallt.2
2/2
1/1
~ $ cat xmb.smallt.2
#!/bin/bash

unset -v success fail cmd a e

 run() {
# uaage , $1 successcase cmd , $2 fail cmd , $3 and further actual cmd to
run
local IFS=' '
eval "${*:3} &"
success[$!]=$1 fail[$!]=$2 cmd[$!]=${*:3}
 }

   wa() {
local pid ret
wait -n -p pid
ret=$?
#printf %s\\n "pid $pid cmd ${me[pid]} returned $ret"
  if (( ret )) && e=${fail[pid]} ; then
 [[ $e ]] &&
eval "$e"
  else
e=${success[pid]}
 [[ $e ]] &&
eval "$e"
  fi
   }

 t2() {
sleep .75
return 3
 }

run 'echo 1/1' 'echo 1/2' sleep 1
run 'echo 2/1' 'echo 2/2' t2

wa
wa

On Mon, Mar 11, 2024, 22:45 alex xmb sw ratchev  wrote:

>
>
> On Mon, Mar 11, 2024, 22:40 alex xmb sw ratchev  wrote:
>
>>
>>
>> On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev 
>> wrote:
>>
>>> ~ $ bash xmb.smallt
>>> pid 14333 cmd t2 returned 3
>>> pid 14332 cmd sleep 1 returned 0
>>>
>>> ~ $ cat xmb.smallt
>>> #!/bin/bash
>>>
>>>  run() {
>>> local IFS=' ' run=$*
>>> eval "$run &"
>>> me[$!]=$run
>>>  }
>>>
>>
>> alternativly to mention aliases usage
>>
>> shopt -s expand_aliases
>>
>> then in func
>> IFS=' '
>> alias torun=$*
>>
>
> and with quotes prolly ye =/
>
> then
>> torun &
>>
>>  wa() {
>>> local pid
>>> wait -n -p pid
>>> printf %s\\n "pid $pid cmd ${me[pid]} returned $?"
>>>  }
>>>
>>>  t2() {
>>> sleep .75
>>> return 3
>>>  }
>>>
>>> run sleep 1
>>> run t2
>>>
>>> wa
>>> wa
>>>
>>> On Mon, Mar 11, 2024, 22:24 Greg Wooledge  wrote:
>>>
 On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote:
 > On Mon, 11 Mar 2024, 21:08 Kerin Millar,  wrote:
 > > The pid can be obtained with the -p option, as of 5.1. Below is a
 > > synthetic example of how it might be put into practice.

 I'd forgotten about that one.  A recent addition, and one I've never
 used
 yet.

 > > #!/bin/bash
 > >
 > > declare -A job_by status_by
 > > max_jobs=4
 > > jobs=0
 > >
 > > wait_next() {
 > > local pid
 > > wait -n -p pid
 > > status_by[$pid]=$?
 > >
 >
 > How exactly is this indexing implemented internally? Does a first
 number on
 > index n take m bits (using a linked list) or does it take n * m bits
 (using
 > realloc(3))?

 "declare -A" makes it an associative array (hash table).

 Without that declare -A, it would be a sparsely indexed array.

 In either case, it doesn't just blindly allocate millions of bytes of
 memory.  It uses something slimmer.




xmb.smallt.2
Description: Binary data