Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-13 Thread Ken Brown
On 3/13/2023 9:15 AM, Paul Smith wrote: On Sun, 2023-03-12 at 11:25 -0400, Ken Brown wrote: If I'm right, the solution would seem to be to disable the use of pselect on Cygwin when the jobserver is using a fifo.  I'll try that on a local build of make and see if I can still reproduce the problem

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-13 Thread Paul Smith
On Sun, 2023-03-12 at 11:25 -0400, Ken Brown wrote: > > If I'm right, the solution would seem to be to disable the use of > > pselect on Cygwin when the jobserver is using a fifo.  I'll try > > that on a local build of make and see if I can still reproduce the > > problem. > > Never mind.  My spe

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-12 Thread Ken Brown
On 3/11/2023 12:51 PM, Ken Brown wrote: Update: The hang occurred again.  It appears to be caused by an infinite loop starting with a call to pselect[*]. [...] If I'm right, the solution would seem to be to disable the use of pselect on Cygwin when the jobserver is using a fifo.  I'll try tha

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-11 Thread Ken Brown
On 3/2/2023 5:39 PM, Ken Brown wrote: I'm returning to this thread because a surprising thing happened.  I decided to try to debug the fifo problem I reported at the beginning of the thread (a hang building TeX Live on Cygwin when the jobserver used a fifo).  So I installed make 4.4.1 built wit

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-02 Thread Ken Brown
I'm returning to this thread because a surprising thing happened. I decided to try to debug the fifo problem I reported at the beginning of the thread (a hang building TeX Live on Cygwin when the jobserver used a fifo). So I installed make 4.4.1 built with fifos enabled (by setting CPPFLAGS=-

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-24 Thread Ken Brown
On 2/24/2023 11:29 AM, Paul Smith wrote: On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: Sorry, I spoke too soon.  I just went back to my original use case, in which I used the TeX Live "Build" script instead of directly invoking make, and I again got the jobserver warning. Thanks for the

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-24 Thread Paul Smith
On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: > Sorry, I spoke too soon.  I just went back to my original use case, > in which I used the TeX Live "Build" script instead of directly > invoking make, and I again got the jobserver warning. Thanks for the info. TL;DR: this is not an error in m

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-24 Thread Ken Brown
On 2/24/2023 9:54 AM, Paul Smith wrote: On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: I'm attaching that script so you can see exactly how "make" is invoked in a subshell.  I'm also attaching my build log up to the point of the warning and the Makefile in the ft-build directory in which th

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-24 Thread Paul Smith
On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: > I'm attaching that script so you can see exactly how "make" is > invoked in a subshell.  I'm also attaching my build log up to the > point of the warning and the Makefile in the ft-build directory in > which the warning occurred.  I've previousl

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-23 Thread Ken Brown
On 2/21/2023 5:59 PM, Ken Brown wrote: On 2/21/2023 1:26 PM, Paul Smith wrote: On Tue, 2023-02-21 at 13:11 -0500, Ken Brown wrote: I think you're on the right track.  I got through 'make -j13 check' without the jobserver warning. w00t!  Thanks for the help.  A full fix should be in 4.4.1 whic

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 1:26 PM, Paul Smith wrote: On Tue, 2023-02-21 at 13:11 -0500, Ken Brown wrote: I think you're on the right track.  I got through 'make -j13 check' without the jobserver warning. w00t! Thanks for the help. A full fix should be in 4.4.1 which I hope to release this week or weekend

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Paul Smith
On Tue, 2023-02-21 at 13:11 -0500, Ken Brown wrote: > I think you're on the right track.  I got through 'make -j13 check' > without the jobserver warning. w00t! Thanks for the help. A full fix should be in 4.4.1 which I hope to release this week or weekend.

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 11:34 AM, Paul Smith wrote: On Tue, 2023-02-21 at 10:36 -0500, Paul Smith wrote: But, I think I might have found the bug.  If I'm right, it's a doozy! But, as you mentioned it's not widespread because it only affects pipe jobservers.  I'm glad 4.4.1 is almost ready to go.  I'll try

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Paul Smith
On Tue, 2023-02-21 at 10:36 -0500, Paul Smith wrote: > But, I think I might have found the bug.  If I'm right, it's a doozy! > But, as you mentioned it's not widespread because it only affects > pipe jobservers.  I'm glad 4.4.1 is almost ready to go.  I'll try to > write a test case for it to see i

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Paul Smith
On Tue, 2023-02-21 at 10:12 -0500, Ken Brown wrote: > > There will likely be a lot of output. > > Here it is. Oops I got the rule wrong for the set -x. But, I think I might have found the bug. If I'm right, it's a doozy! But, as you mentioned it's not widespread because it only affects pipe jo

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 9:39 AM, Paul Smith wrote: On Tue, 2023-02-21 at 08:35 -0500, Ken Brown wrote: make  check-TESTS make[4]: Entering directory '/home/kbrown/src/texlive/test.x86_64/Work/texk/kpathsea' make[5]: Entering directory '/home/kbrown/src/texlive/test.x86_64/Work/texk/kpathsea' OK, I see th

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Paul Smith
On Tue, 2023-02-21 at 08:35 -0500, Ken Brown wrote: > make  check-TESTS > make[4]: Entering directory > '/home/kbrown/src/texlive/test.x86_64/Work/texk/kpathsea' > make[5]: Entering directory > '/home/kbrown/src/texlive/test.x86_64/Work/texk/kpathsea' OK, I see the rule that generates the [5] re

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Paul Smith
On Tue, 2023-02-21 at 08:35 -0500, Ken Brown wrote: > On 2/21/2023 3:54 AM, Ken Brown wrote: > > On 2/21/2023 12:27 AM, Paul Smith wrote: > > > Just to note, I do run the regression test suite with the > > > equivalent of "jobserver-style=pipe" (basically I force the > > > configure to believe that

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 12:27 AM, Paul Smith wrote: On Mon, 2023-02-20 at 20:21 -0500, Ken Brown wrote: Parallel make is still not working reliably.  I've just discovered that my TeX Live build logs have several occurrences of the following warning:    jobserver unavailable: using -j1.  Add '+' to parent

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-20 Thread Paul Smith
On Mon, 2023-02-20 at 20:21 -0500, Ken Brown wrote: > Parallel make is still not working reliably.  I've just discovered > that my TeX Live build logs have several occurrences of the following > warning: > >    jobserver unavailable: using -j1.  Add '+' to parent make rule. > > This has been goin

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-20 Thread Ken Brown
On 2/19/2023 9:29 AM, Paul Smith wrote: I will change the default setting of the jobserver to use "pipe" on Cygwin, at least for now. Parallel make is still not working reliably. I've just discovered that my TeX Live build logs have several occurrences of the following warning: jobserver

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Ken Brown
On 2/19/2023 9:29 AM, Paul Smith wrote: On Sun, 2023-02-19 at 09:17 -0500, Ken Brown wrote: So I'm not sure where to go from here on Cygwin.  Should I force Cygwin builds to use the "pipe" jobserver, as I've done for GNU/Hurd? My preference would be for you to provide a configure option, which

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Paul Smith
On Sun, 2023-02-19 at 09:17 -0500, Ken Brown wrote: > > So I'm not sure where to go from here on Cygwin.  Should I force > > Cygwin builds to use the "pipe" jobserver, as I've done for > > GNU/Hurd? > > My preference would be for you to provide a configure option, which > defaults to "pipe" on Cy

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Ken Brown
On 2/19/2023 8:49 AM, Paul Smith wrote: On Wed, 2023-02-15 at 13:57 -0500, Ken Brown wrote: One thing to keep in mind here is that your tests on Cygwin were done on Cygwin 2.9.0, in which FIFOs were very poorly supported.  For example, a FIFO couldn't have multiple readers or writers.  If GNU/Hu

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Paul Smith
On Wed, 2023-02-15 at 13:57 -0500, Ken Brown wrote: > One thing to keep in mind here is that your tests on Cygwin were done > on Cygwin 2.9.0, in which FIFOs were very poorly supported.  For > example, a FIFO couldn't have multiple readers or writers.  If > GNU/Hurd has similar limitations, that co

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Paul Smith
On Sun, 2023-02-19 at 12:44 +0100, Bruno Haible wrote: >   - 4 failures in category 'features/jobserver' >   - 2 failure in category 'functions/shell' I looked at the error logging you sent, and I think these are just a result of an incomplete/incorrect setting of the default value to "pipe". Hop

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Bruno Haible
Paul Smith wrote: > I've made a change that causes GNU/Hurd to not use the mkfifo()-based > jobserver, and go back to using pipe(). That's nice, because compared to the previous results - 4 failures in category 'features/archives', due to "cc: not found". - 5 failures in category 'features/jo

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-18 Thread Paul Smith
On Wed, 2023-02-15 at 13:57 -0500, Ken Brown wrote: > If I simply run 'make -j13 world' without using the Build script, it > doesn't hang.  I don't know if this provides any clue as to where the > problem is. Speaking for myself, it doesn't give me any clues. I don't see why it should matter. Bu

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-18 Thread Paul Smith
On Wed, 2023-02-15 at 19:34 +0100, Bruno Haible wrote: > > I may be misremembering but I thought that you had tried forcing > > the pipe jobserver option on GNU/Hurd and it didn't help. > > This must be a misunderstanding. I never said that. I don't even know > how to do this, within the framework

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-15 Thread Ken Brown
On 2/15/2023 1:34 PM, Bruno Haible wrote: Paul Smith wrote: And possibly also on GNU/Hurd. Cf. https://lists.gnu.org/archive/html/bug-make/2023-01/msg00107.html I may be misremembering but I thought that you had tried forcing the pipe jobserver option on GNU/Hurd and it didn't help. This mus

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-15 Thread Bruno Haible
Paul Smith wrote: > > And possibly also on GNU/Hurd. Cf. > > https://lists.gnu.org/archive/html/bug-make/2023-01/msg00107.html > > I may be misremembering but I thought that you had tried forcing the > pipe jobserver option on GNU/Hurd and it didn't help. This must be a misunderstanding. I never

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-15 Thread Paul Smith
On Tue, 2023-02-14 at 01:04 +0100, Bruno Haible wrote: > And possibly also on GNU/Hurd. Cf. > https://lists.gnu.org/archive/html/bug-make/2023-01/msg00107.html I may be misremembering but I thought that you had tried forcing the pipe jobserver option on GNU/Hurd and it didn't help.

Regression on Cygwin: Problems with parallel make in 4.4

2023-02-14 Thread Ken Brown
Several packages that used to build fine on Cygwin with parallel make now fail to build. I either get strange errors or a hang. The problems started with version 4.4 and seem to stem from the fact that make now uses a FIFO by default instead of a pipe. If I use the flag '--jobserver-style=pi

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-13 Thread Bruno Haible
Ken Brown wrote: > I suggest > that you provide a configure option to set the jobserver style at the > time make is built, and set it to 'pipe' by default on Cygwin. And possibly also on GNU/Hurd. Cf. https://lists.gnu.org/archive/html/bug-make/2023-01/msg00107.html Bruno