Re: Parallel builds across makefiles

2016-08-04 Thread David Boyce
see it asymptotically approach > the no-parallel-building time. But to see it repeatably much longer just > seems a bit bizarre. > > > > *From:* Brian Vandenberg [mailto:phant...@gmail.com] > *Sent:* Wednesday, July 27, 2016 4:59 PM > *To:* Gardell, Steven > *Cc:* Bug-ma

RE: Parallel builds across makefiles

2016-07-30 Thread Gardell, Steven
] Sent: Wednesday, July 27, 2016 4:59 PM To: Gardell, Steven Cc: Bug-make@gnu.org Subject: Re: Parallel builds across makefiles I haven't seen similar issues but I have a hypothesis: make is single-threaded and therefore it consumes output from jobs in the same thread it uses to reap/spawn new

Re: Parallel builds across makefiles

2016-07-27 Thread Brian Vandenberg
I haven't seen similar issues but I have a hypothesis: make is single-threaded and therefore it consumes output from jobs in the same thread it uses to reap/spawn new jobs. If make is spending a large enough amount of time consuming output then this will impacts the rate at which it can spawn/reap

Re: Parallel builds across makefiles

2016-07-25 Thread David Boyce
ariable was not set. I set it to /tmp to insure that it is on > a physical device. (The build and our home directories are NFS mounted) The > total elapsed time for the “-Otarget” run was then 6 hours. This is enough > of a reduction to be a little suspicious, but not much more. &g

RE: Parallel builds across makefiles

2016-07-25 Thread Gardell, Steven
Cc: psm...@gnu.org<mailto:psm...@gnu.org>; Bug-make@gnu.org<mailto:Bug-make@gnu.org> Subject: Re: Parallel builds across makefiles A couple of suggestions: 1. Check your value of TMPDIR if any. All -O is doing is redirecting output into a temp file and dumping it later. Effectively i

Re: Parallel builds across makefiles

2016-07-25 Thread David Boyce
on’t think there is a smoking gun > here. > > > > I have also run the “-Otrace” case at a few different job-levels: > > > > j8 – 6:34 > > j32 – 6:00 > > j256 – 9:49 > > > > *From:* David Boyce [mailto:david.s.bo...@gmail.com] > *Sent:* Sunday, J

RE: Parallel builds across makefiles

2016-07-22 Thread Gardell, Steven
a smoking gun here. I have also run the “-Otrace” case at a few different job-levels: j8 – 6:34 j32 – 6:00 j256 – 9:49 From: David Boyce [mailto:david.s.bo...@gmail.com] Sent: Sunday, July 17, 2016 5:03 PM To: Gardell, Steven Cc: psm...@gnu.org; Bug-make@gnu.org Subject: Re: Parallel builds a

Re: Parallel builds across makefiles

2016-07-17 Thread David Boyce
> the testing is with -Otarget. But I have tried all of them. > > -Original Message- > From: Paul Smith [mailto:psm...@gnu.org] > Sent: Sunday, July 17, 2016 3:14 PM > To: Gardell, Steven ; Bug-make@gnu.org > Subject: Re: Parallel builds across makefiles > > On Sun

RE: Parallel builds across makefiles

2016-07-17 Thread Gardell, Steven
; Bug-make@gnu.org Subject: Re: Parallel builds across makefiles On Sun, 2016-07-17 at 15:48 +, Gardell, Steven wrote: > OK. Thanks! I will try the latest version. > > FWIW, I am measuring is total wall clock time to complete the build. > (date; gmake... ; date) This goes up substa

Re: Parallel builds across makefiles

2016-07-17 Thread Paul Smith
On Sun, 2016-07-17 at 15:48 +, Gardell, Steven wrote: > OK. Thanks! I will try the latest version.  > > FWIW, I am measuring is total wall clock time to complete the build. > (date; gmake... ; date) This goes up substantially with all sync modes > other than "none" if gmake has parallel invoca

RE: Parallel builds across makefiles

2016-07-17 Thread Gardell, Steven
To: Gardell, Steven ; Bug-make@gnu.org Subject: Re: Parallel builds across makefiles On Thu, 2016-07-14 at 15:40 -0700, sgardell wrote: > But when I try to build across makefiles in parallel then I see two > different things: >    1) If I turn on any sort of output sync it gets dramatical

Re: Parallel builds across makefiles

2016-07-17 Thread Paul Smith
On Thu, 2016-07-14 at 15:40 -0700, sgardell wrote: > But when I try to build across makefiles in parallel then I see two > different things: >    1) If I turn on any sort of output sync it gets dramatically slower. > Sometimes slower than our -j1 time! I recommend trying with the latest release, G

Re: Parallel builds across makefiles

2016-07-15 Thread sgardell
seeing know/expected? -- View this message in context: http://gnu-make.2324884.n4.nabble.com/Parallel-builds-across-makefiles-tp17266p17269.html Sent from the Gnu - Make - Bugs mailing list archive at Nabble.com. ___ Bug-make mailing list Bug-make@gnu.org

Parallel builds across makefiles

2016-07-14 Thread thutt
sgardell writes: > We are using a standard distro of gmake 4.1 on RH 6.4. > > We have a fairly well structured makefile system that leverages the ACE make > framework. A linear build (-j1) is about 3 hours. Running a parallel build > that builds leaf nodes (libraries) in parallel is about 1 ho

Parallel builds across makefiles

2016-07-14 Thread sgardell
'top' during the "highly parallelized" runs shows the system essentially idle. -- View this message in context: http://gnu-make.2324884.n4.nabble.com/Parallel-builds-across-makefiles-tp17266.html Sent from the Gnu - Make - Bugs mailing list a