[PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-26 Thread Stefan Beller
Similar to `git submodule foreach` the new command `git submodule foreach_parallel` will run a command on each submodule. The commands are run in parallel up to the number of cores by default, or you can specify '-j 4' tun just run with 4 threads for example. One major difference to `git submodul

Re: [PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-26 Thread Stefan Beller
On Wed, Aug 26, 2015 at 10:06 AM, Jeff King wrote: > On Tue, Aug 25, 2015 at 10:28:24AM -0700, Stefan Beller wrote: > >> +int module_foreach_parallel(int argc, const char **argv, const char *prefix) >> +{ >> [...] >> + for (i = 0; i < ce_used; i++) { >> + const struct submodule *su

Re: [PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-26 Thread Jeff King
On Tue, Aug 25, 2015 at 10:28:24AM -0700, Stefan Beller wrote: > +int module_foreach_parallel(int argc, const char **argv, const char *prefix) > +{ > [...] > + for (i = 0; i < ce_used; i++) { > + const struct submodule *sub; > + const struct cache_entry *ce = ce_entries

Re: [PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-25 Thread Junio C Hamano
Junio C Hamano writes: >> Why would we want to unplug the task queue from somewhere else? > > When you have a dispatcher more intelligent than a stupid FIFO, I > would imagine that you would want to be able to do this pattern, > especially when coming up with a task (not performing a task) takes

Re: [PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-25 Thread Junio C Hamano
Stefan Beller writes: >>> + while (1) { >>> + ssize_t len = xread(cp->err, buf, sizeof(buf)); >>> + if (len < 0) >>> + die("Read from child failed"); >>> + else if (len == 0) >>> + break; >>> + else { >>>

Re: [PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-25 Thread Stefan Beller
On Tue, Aug 25, 2015 at 2:09 PM, Junio C Hamano wrote: > Stefan Beller writes: > >> This runs a command on each submodule in parallel and should eventually >> replace `git submodule foreach`. >> >> There is a new option -j/--jobs (inspired by make) to specify the number >> of parallel threads. >>

Re: [PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-25 Thread Junio C Hamano
Stefan Beller writes: > This runs a command on each submodule in parallel and should eventually > replace `git submodule foreach`. > > There is a new option -j/--jobs (inspired by make) to specify the number > of parallel threads. > > The jobs=1 case needs to be special cases to exactly replicate

[PATCH 3/5] submodule: helper to run foreach in parallel

2015-08-25 Thread Stefan Beller
This runs a command on each submodule in parallel and should eventually replace `git submodule foreach`. There is a new option -j/--jobs (inspired by make) to specify the number of parallel threads. The jobs=1 case needs to be special cases to exactly replicate the current default behavior of `gi