Parallelization of shell scripts for 'configure' etc.

2022-06-13 Thread Paul Eggert
In many Gnu projects, the 'configure' script is the biggest barrier to building because it takes s long to run. Is there some way that we could improve its performance without completely reengineering it, by improving Bash so that it can parallelize 'configure' scripts? For ideas about thi

Re: Parallelization of shell scripts for 'configure' etc.

2022-06-13 Thread Dale R. Worley
Paul Eggert writes: > In many Gnu projects, the 'configure' script is the biggest barrier to > building because it takes s long to run. Is there some way that we > could improve its performance without completely reengineering it, by > improving Bash so that it can parallelize 'configure' s

Re: Parallelization of shell scripts for 'configure' etc.

2022-06-13 Thread Paul Eggert
On 6/13/22 18:25, Dale R. Worley wrote: It seems to me that bash provides the needed tools -- "( ... ) &" lets you run things in parallel. Similarly, if you've got a lot of small tasks with a complex web of dependencies, you can encode that in a "makefile". It seems to me that the heavy work is

Re: Parallelization of shell scripts for 'configure' etc.

2022-06-13 Thread Alex Ameen
Yeah honestly splitting most of the `configure` checks into multiple threads is definitely possible. Caching between projects is even a straightforward extension with systems like `Nix`. The "gotcha" here in both cases is that existing scripts that are living in source tarballs are not feasible t

Re: Parallelization of shell scripts for 'configure' etc.

2022-06-13 Thread Alex Ameen
You can try to use the `requires` toposort routine to identify "Strongly Connected Sub-Components", which is where I imagine you'll get the best results. What you'll need to watch out for is undeclared ordering requirements that parallelism would break. The `m4sh` and `m4sugar` source code is docu