Re: [PATCH] src/main.c: Add -J to detect number of job slots based on nproc.

> default, > if there was to be one at all, should be conservative so nproc * 2 would be a > bad choice. I've always used nproc * 1.5. This factor was determined empirically by observing idle time on many builds using nproc * 1 and nproc * 2. -- -- Howard Chu CTO, Symas Cor

Re: Add support for limiting CPU pressure

ry will Just Work after that, and all of the relevant OSs have support for named pipes. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Re: Goodbye to GNU make's "build.sh" ... ?

re capable of supporting gmake that don't already have a make utility. Or that telling anyone so unlucky "first bootstrap with gmake 4.3" is a bad answer. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Re: Parallel job instance identifiers?

Paul Smith wrote: > On Fri, 2021-08-13 at 14:14 +0100, Howard Chu via Bug reports and > discussion for GNU make wrote: >> I'm looking for a way to expose a job ID number to the individual >> jobs, in a consecutive range from 1-[number of jobs]. Not just >> unique nu

Re: Parallel job instance identifiers?

he range of 9000-9020. On a build/test machine we don't worry about conflicts, a developer has no business running conflicting software at the same time as building/testing our software. And of course, the baseport 9000 can be moved at will. > > Christian > > On Fri, Aug 13

Parallel job instance identifiers?

er of client/server test scripts in parallel. Is there anything that could do this now, exposed as a Make variable? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Re: "make -jN" requires mechanical changes to a Makefile [SOLVED]

all: copy1 copy2 copy3 copy4 copy1: Makefile install -c -m 644 Makefile $@ copy2: copy1 install -c -m 644 Makefile $@ copy3: copy2 install -c -m 644 Makefile $@ copy4: copy3 install -c -m 644 Makefile $@ ===

Re: [bug #56701] Do not allow -j without a number

ssue and avoid introducing any regressions. > > For example, if -j is given without an argument and no -l option is provided, > behave as if -j -l $(nproc) was specified. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highl

Re: Should make handle non-blocking jobserver and use socket pair instead?

s actually a socket pair instead of a pipe, but then > that jobserver needs to be created externally, as the GNU make > creating the jobserver creates a pipe and not a socket pair. As a > hack, a wrapper program could do it and then call GNU make. > > What are your opinions o

Re: "make -jN" requires mechanical changes to a Makefile

-m 644 Makefile copy3 >> install -c -m 644 Makefile copy4 > > I think the "representative" file should be copy4 here, because it's the one > that gets created last. That sort of thing is only true in serial make, you can't rely on it in parallel make. -- --

Re: Parallel Build with GNU make

an error in your Makefile. Read this: http://highlandsun.com/hyc/#Make in particular this http://highlandsun.com/hyc/GNUYou.htm -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Archite

Re: Dynamic adjustments of build dependencies for the number of available processors

itten Makefile, your serial preparation step will remain serial no matter what "-j" arguments are used, so it should be safe to just invoke "make -j xx" at the beginning and let things run serially or in parallel as necessary. -- -- Howard Chu CTO, Symas Corp. h

Re: feature request: parallel builds feature

stdout. That has nothing to do with stdin. Looks to me like you have no F'ing idea what you're talking about. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.o

Re: feature request: parallel builds feature

ement, but my suspicion is that all such possible use-cases are better solved by a change of process at a level above what make can provide. _______ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make -- -- Howard Chu CTO, Symas

Re: Thoughts on limiting the parallel build load, suggestion for a new "-j auto" option

achine so heavily as to interfere with browsing, reading mail, playing nethack and other things developers like to still be able to do while waiting for a build. Eddy. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.

Re: [bug #35485] New $(.MFDIR) built-in variable

etc. because those functions must first chop the string up into words, then operate on those, and that happens every time the function is invoked. If that happens a lot, then you should simply construct the lists once and keep them... -- -- Howard Chu CTO, Symas Corp. http://ww

Re: Thoughts on limiting the parallel build load, suggestion for a new "-j auto" option

piles I find that CPU count * 1.5 works well. Some people use make for things other than invoking compilers though, and some compilers have different resource requirements than a typical gcc run, so even that isn't a widely applicable estimate. Only the end user can determine the best val

Re: New Feature Submission for GNU Make

rom leveraging parallel make, so that's another reason not to do things that way... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ ___

Re: New Feature Submission for GNU Make

rees, where "foo/bar/.." != "foo". -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ ___ Bug-

Re: patch to support output synchronization under -j

cores, trying to run with -j8192 or somesuch. (Default pipe buffer is 4K which sets -j4096 as the practical limit without mucking with kernel params.) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/

Re: patch to support output synchronization under -j

ou wouldn't just use fstat() and compare dev and ino numbers? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Re: Fwd: [RFC]serialize the output of parallel make?

ere's no reason that it can't be fixed to not mingle/mangle lines from different processes together. But again, that's not gnu-make's problem, that's a Cygwin issue. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun htt

Re: Fwd: [RFC]serialize the output of parallel make?

erfect world -l would be useful. In fact, since load averages are calculated so slowly, by the time your -l limit is reached the actual CPU load will have blown past it and your machine will be thrashing. That's the entire reason I came up with the -j implementation in the first place. --

Re: Fwd: [RFC]serialize the output of parallel make?

my experience, any more than that and you're losing too much CPU to scheduling overhead instead of real work. Any less and you're giving up too much in idle or I/O time. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlands

Re: [RFC]serialize the output of parallel make?

Chiheng Xu wrote: On Fri, Jul 30, 2010 at 6:01 PM, Howard Chu wrote: Chiheng Xu wrote: On Fri, Jul 30, 2010 at 5:35 PM, Eli Zaretskiiwrote: I asked for an example. Could you please show a "messy" output and the output you'd like to have after "serialization&quo

Re: [RFC]serialize the output of parallel make?

ines of multi-line output. stderr may skew things since it's usually nonbuffered, but again, that's not the common case. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/

Re: [RFC]serialize the output of parallel make?

of off the top of my head. It certainly does not mean that it would not be a good thing to have this ability though. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.

[bug #28134] inconsistent reporting of linux kernel builds with -j2

Follow-up Comment #3, bug #28134 (project make): As a complete digression, and based on 26 years of doing parallel builds (damn it's been a long time), I just wanted to point out that you can generally use 50% more jobs than your number of CPUs/cores. E.g., for a dual-core machine, use make -j3.

[bug #22923] option to prevent "interspersed" output in parallel builds

Follow-up Comment #7, bug #22923 (project make): Reaching back 25 years again, this is basically what Alliant Concentrix parallel make did. It prefixed each output line with the job number |1|, |2| and so on. (This is also why in my original proposal for make -j, I used different numbered job tok

Re: [bug #22923] option to prevent "interspersed" output in parallel builds

rver) also redirected all of the spawned processes' stdio to a set of pipes, so that all of their output was prefixed with |jobnum|. It was quite nice, but I never felt it was important enough to duplicate that behavior for gmake. -- -- Howard Chu CTO, Symas Corp. http:

Re: Switching from CVS to GIT

27;re building Windows-native code with it doesn't really enter the equation. Aside from that, happy to leave this for later. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc/ Chief Architect, Open

Re: EAGAIN in "read jobs pipe"

arily use maybe -j 10 (when I know 10 more calls to make are made) On some machines this might be unacceptable and bring them to their knees. On other machines it can work really well. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun

Re: EAGAIN in "read jobs pipe"

, but obviously the right balance depends on your disk speeds relative to your CPUs... The fact that your machine stays responsive during a build does not necessarily mean that you will gain any benefit from increasing the -j number... When I see that total CPU idle time is about 3% or

Re: [bug #18396] stack size setrlimit call interacts badly with Solaris/x86 kernel bug

eral, Unix doesn't impose restrictions retroactively. (E.g., if you open a file and have a valid descriptor on it, and someone else chmods the file, removing your access permissions, you still have your original privileges through the open descriptor.) -- -- Howard Chu Chief Architect, Sy

Re: [bug #18396] stack size setrlimit call interacts badly with Solaris/x86 kernel bug

allocs are good, but stack-based allocation is still better a lot of the time. Especially for temporary variables that are just going to be discarded after a few computations. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlands

Re: [bug #17873] .NOTPARALLEL enhancements

omething interesting that I couldn't live without, and it was worth my effort to push it forward. Frankly this is trivial and personally uninteresting, and I can live without it just fine, since I know how to write correct Makefiles. -- -- Howard Chu Chief Architect, Symas Corp. http:

Re: [bug #17873] .NOTPARALLEL enhancements

earth would anyone want to do that!?) I've noticed that SCons has this feature, but it might come in handy for verifying that a makefile is "properly written." Yes, I've suggested this a number of times in the past. Never formally in a bug report, but #17881 which I just s

[bug #17881] Better documentation of make rules

Follow-up Comment #1, bug #17881 (project make): It might be worthwhile to add some kind of makefile validation flag, which processes all prerequisites in random or right-to-left order, specifically to expose occurrences of incorrect prerequisites. ___

Re: [bug #17881] Better documentation of make rules

Bah. Pretty sure I selected "Documentation" for this submission, not "Enhancement". Howard Chu wrote: URL: <http://savannah.gnu.org/bugs/?17881> Summary: Better documentation of make rules Project: make Submitted by: hy

[bug #17881] Better documentation of make rules

URL: Summary: Better documentation of make rules Project: make Submitted by: hyc Submitted on: Thursday 09/28/2006 at 20:26 Severity: 3 - Normal Item Group: Enhancement

[bug #17873] .NOTPARALLEL enhancements

Follow-up Comment #3, bug #17873 (project make): Haven't been paying a lot of attention, but when is such a lame thing nas .NOTPARALLEL needed? Why aren't people just writing proper dependencies so that parallel/serial is irrelevant? ___ R

Re: Error in processing of :: targets

CTED]>; sleep 1; echo [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> all.parallel: aaa bbb ccc aaa bbb ccc: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> -- -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://w

Re: parallelism: job slots vs. levels

rack about how to use -j. Since your two examples have nothing to do with each other, I don't see how you can reach this conclusion. -- -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: P

Re: parallelism: job slots vs. levels

ile is broken. The point is that in a correctly written Makefile with all dependency relationships explicitly spelled out, there will be no difference whether make is run serially or in parallel. If there is any difference, then the Makefile rules didn't correctly capture all of the real

RE: DJGPP MAKE.EXE - CANNOT PRINT HELP SCREEN

completely different animal anyway. -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: Premier OpenSource Development and Support > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PR

RE: make-3.79 on solaris8 broken

don't mention stat() as being one of the restartable system calls. (But I'd bet that it is...) -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc Symas: Premier OpenSource Development and Support

RE: Recursive GMAKE with -j option

bash will probably work fine. On most Linux systems, bash is the *only* "POSIX" shell anyway. -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc > -Original Message- > From:

RE: How-to

oceedings of the Ninth Annual Sun Users Group Conference, December 1991. (The title is a play on "make world" which I had been repeatedly been doing while compiling X11, and getting really really sick of imake. I completely ripped imake out of the X11 source tree and replaced everything with

RE: GNU enhancement -- $(xargs cmd, list)

I would omit the XARGS LEN. Presumably the local platform's xargs command is already properly configured for the native system's line length limitations. -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlands

RE: GNU enhancement -- $(xargs cmd, list)

my_exec -command my_cfg_file (Copy temp command lines into my_cfg_file to make it, then use it in stage2...) This still looks like a job best left to echo. I suppose you could also use the commandline "printf" command as well, if you have it and are facing a quoting nightmare.

RE: GNU enhancement -- $(xargs cmd, list)

Try "xargs command < tmpfile" instead of "cat tmpfile | xargs command" ... -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc > -Original Message- > From: [EM

RE: GNU enhancement -- $(xargs cmd, list)

problem mechanics were understood... -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of > Reid Madsen >

RE: does gnumake 3.77 wildcard work under solaris 2.6?

This is a known bug in 3.77. 3.79 is the current version, you really need to update. -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc > -Original Message- > From: John Lewis 425-865-3510 [

RE: -j fails on DYNIX/ptx

file does not exist, even though it does. The big question is what is generating the signals - presumably it's SIGCHLD in this case, and there's probably not much you can do about that. -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www

Re: Targets not being updated

E) in the gdbm/libgdbm.la rule. Right now this is pretty messy, and there's nothing here that explicitly states where gdbm/gdbm.h came from. -- Howard Chu Chief Architect, Symas Corp. Director, Highland Sun http://www.symas.com http://highlandsun.com/hyc