Re: Parallel builds across makefiles

2016-08-04 Thread David Boyce
see it asymptotically approach > the no-parallel-building time. But to see it repeatably much longer just > seems a bit bizarre. > > > > *From:* Brian Vandenberg [mailto:phant...@gmail.com] > *Sent:* Wednesday, July 27, 2016 4:59 PM > *To:* Gardell, Steven > *Cc:* Bug-ma

RE: Parallel builds across makefiles

2016-07-30 Thread Gardell, Steven
] Sent: Wednesday, July 27, 2016 4:59 PM To: Gardell, Steven Cc: Bug-make@gnu.org Subject: Re: Parallel builds across makefiles I haven't seen similar issues but I have a hypothesis: make is single-threaded and therefore it consumes output from jobs in the same thread it uses to reap/spawn new

Re: Parallel builds across makefiles

2016-07-27 Thread Brian Vandenberg
I haven't seen similar issues but I have a hypothesis: make is single-threaded and therefore it consumes output from jobs in the same thread it uses to reap/spawn new jobs. If make is spending a large enough amount of time consuming output then this will impacts the rate at which it can spawn/reap

Re: Parallel builds across makefiles

2016-07-25 Thread David Boyce
ariable was not set. I set it to /tmp to insure that it is on > a physical device. (The build and our home directories are NFS mounted) The > total elapsed time for the “-Otarget” run was then 6 hours. This is enough > of a reduction to be a little suspicious, but not much more. &g

RE: Parallel builds across makefiles

2016-07-25 Thread Gardell, Steven
Cc: psm...@gnu.org<mailto:psm...@gnu.org>; Bug-make@gnu.org<mailto:Bug-make@gnu.org> Subject: Re: Parallel builds across makefiles A couple of suggestions: 1. Check your value of TMPDIR if any. All -O is doing is redirecting output into a temp file and dumping it later. Effectively i

Re: Parallel builds across makefiles

2016-07-25 Thread David Boyce
on’t think there is a smoking gun > here. > > > > I have also run the “-Otrace” case at a few different job-levels: > > > > j8 – 6:34 > > j32 – 6:00 > > j256 – 9:49 > > > > *From:* David Boyce [mailto:david.s.bo...@gmail.com] > *Sent:* Sunday, J

RE: Parallel builds across makefiles

2016-07-22 Thread Gardell, Steven
a smoking gun here. I have also run the “-Otrace” case at a few different job-levels: j8 – 6:34 j32 – 6:00 j256 – 9:49 From: David Boyce [mailto:david.s.bo...@gmail.com] Sent: Sunday, July 17, 2016 5:03 PM To: Gardell, Steven Cc: psm...@gnu.org; Bug-make@gnu.org Subject: Re: Parallel builds a

Re: Parallel builds across makefiles

2016-07-17 Thread David Boyce
> the testing is with -Otarget. But I have tried all of them. > > -Original Message- > From: Paul Smith [mailto:psm...@gnu.org] > Sent: Sunday, July 17, 2016 3:14 PM > To: Gardell, Steven ; Bug-make@gnu.org > Subject: Re: Parallel builds across makefiles > > On Sun

RE: Parallel builds across makefiles

2016-07-17 Thread Gardell, Steven
; Bug-make@gnu.org Subject: Re: Parallel builds across makefiles On Sun, 2016-07-17 at 15:48 +, Gardell, Steven wrote: > OK. Thanks! I will try the latest version. > > FWIW, I am measuring is total wall clock time to complete the build. > (date; gmake... ; date) This goes up substa

Re: Parallel builds across makefiles

2016-07-17 Thread Paul Smith
On Sun, 2016-07-17 at 15:48 +, Gardell, Steven wrote: > OK. Thanks! I will try the latest version.  > > FWIW, I am measuring is total wall clock time to complete the build. > (date; gmake... ; date) This goes up substantially with all sync modes > other than "none" if gmake has parallel invoca

RE: Parallel builds across makefiles

2016-07-17 Thread Gardell, Steven
To: Gardell, Steven ; Bug-make@gnu.org Subject: Re: Parallel builds across makefiles On Thu, 2016-07-14 at 15:40 -0700, sgardell wrote: > But when I try to build across makefiles in parallel then I see two > different things: >    1) If I turn on any sort of output sync it gets dramatical

Re: Parallel builds across makefiles

2016-07-17 Thread Paul Smith
On Thu, 2016-07-14 at 15:40 -0700, sgardell wrote: > But when I try to build across makefiles in parallel then I see two > different things: >    1) If I turn on any sort of output sync it gets dramatically slower. > Sometimes slower than our -j1 time! I recommend trying with the latest release, G

Re: Parallel builds across makefiles

2016-07-15 Thread sgardell
The gmake documentation states clearly that there is no build-time impact from output synchronization. So it seems that the behavior I am seeing in that regard is a gmake bug. Beyond that, how can I actually achieve effective parallelism with nested makefile invocations? Is the behavior I am seeing