Re: AIX and Interix also do early PID recycling.

2012-09-05 Thread Chet Ramey
On 8/28/12 3:21 AM, Roman Rakus wrote: > The attached patch adds ability to set forced number of remembered > statuses, easily during configure time, example: > ./configure --enable-fixedjsmax=2048 > It will force bash to remember 2048 statuses, independently on childmax value. > Patch should cle

Re: AIX and Interix also do early PID recycling.

2012-08-29 Thread Chet Ramey
On 8/29/12 10:06 AM, Michael Haubenwallner wrote: >> I prepared a patch which add configure option to enable and set the number >> of job exit statuses to remember. > > Why not simply use the static CHILD_MAX value instead? > Feels like this is what the spec means - and conforming kernels do not

Re: AIX and Interix also do early PID recycling.

2012-08-29 Thread Roman Rakus
On 08/29/2012 04:06 PM, Michael Haubenwallner wrote: On 08/28/2012 09:21 AM, Roman Rakus wrote: On 08/01/2012 03:13 PM, Chet Ramey wrote: On 7/30/12 10:41 AM, Roman Rakus wrote: Hmm... I don't know much about boundaries of maximum number of user processes. But anyway - do you think that (re)c

Re: AIX and Interix also do early PID recycling.

2012-08-29 Thread Michael Haubenwallner
On 08/28/2012 09:21 AM, Roman Rakus wrote: > On 08/01/2012 03:13 PM, Chet Ramey wrote: >> On 7/30/12 10:41 AM, Roman Rakus wrote: >> >>> Hmm... I don't know much about boundaries of maximum number of user >>> processes. But anyway - do you think that (re)changing js.c_childmax (when >>> `ulimit -u

Re: AIX and Interix also do early PID recycling.

2012-08-28 Thread Roman Rakus
On 08/01/2012 03:13 PM, Chet Ramey wrote: On 7/30/12 10:41 AM, Roman Rakus wrote: Hmm... I don't know much about boundaries of maximum number of user processes. But anyway - do you think that (re)changing js.c_childmax (when `ulimit -u' is changed) is not good? Maybe it's ok up to some fixed u

Re: AIX and Interix also do early PID recycling.

2012-08-20 Thread Michael Haubenwallner
On 07/29/2012 12:46 AM, Chet Ramey wrote: > On 7/27/12 9:50 AM, Michael Haubenwallner wrote: > >> With attached patch I haven't been able to break the testcase below so far >> on that AIX 6.1 box here. >> >> But still, the other one using the $()-childs still fails. > > Try the attached patch f

Re: AIX and Interix also do early PID recycling.

2012-08-01 Thread Chet Ramey
On 7/30/12 10:41 AM, Roman Rakus wrote: > Hmm... I don't know much about boundaries of maximum number of user > processes. But anyway - do you think that (re)changing js.c_childmax (when > `ulimit -u' is changed) is not good? Maybe it's ok up to some fixed upper bound. But if you're going to hav

Re: AIX and Interix also do early PID recycling.

2012-07-30 Thread Roman Rakus
On 07/25/2012 07:12 PM, Chet Ramey wrote: On 7/25/12 10:57 AM, Roman Rakus wrote: And there is also a problem when you change the `ulimit -u' value. See https://bugzilla.redhat.com/show_bug.cgi?id=832997 It's not clear to me that it's desirable to attempt to remember a potentially unlimited n

Re: AIX and Interix also do early PID recycling.

2012-07-28 Thread Chet Ramey
On 7/27/12 9:50 AM, Michael Haubenwallner wrote: > With attached patch I haven't been able to break the testcase below so far > on that AIX 6.1 box here. > > But still, the other one using the $()-childs still fails. Try the attached patch for that. Chet -- ``The lyf so short, the craft so lon

Re: AIX and Interix also do early PID recycling.

2012-07-27 Thread Chet Ramey
On 7/26/12 5:37 PM, Michael Haubenwallner wrote: > > On 07/26/12 20:29, Chet Ramey wrote: >> OK, we have some data, we have a hypothesis, and we have a way to test it. >> Let's test it. >> >> Michael, please apply the attached patch, disable RECYCLES_PIDS, and run >> your tests again. This makes

Re: AIX and Interix also do early PID recycling.

2012-07-27 Thread Chet Ramey
On 7/26/12 5:37 PM, Michael Haubenwallner wrote: > > On 07/26/12 20:29, Chet Ramey wrote: >> OK, we have some data, we have a hypothesis, and we have a way to test it. >> Let's test it. >> >> Michael, please apply the attached patch, disable RECYCLES_PIDS, and run >> your tests again. This makes

Re: AIX and Interix also do early PID recycling.

2012-07-27 Thread Michael Haubenwallner
On 07/26/2012 11:37 PM, Michael Haubenwallner wrote: > On 07/26/12 20:29, Chet Ramey wrote: >> OK, we have some data, we have a hypothesis, and we have a way to test it. >> Let's test it. >> >> Michael, please apply the attached patch, disable RECYCLES_PIDS, and run >> your tests again. This make

Re: AIX and Interix also do early PID recycling.

2012-07-26 Thread Michael Haubenwallner
On 07/26/12 20:29, Chet Ramey wrote: OK, we have some data, we have a hypothesis, and we have a way to test it. Let's test it. Michael, please apply the attached patch, disable RECYCLES_PIDS, and run your tests again. This makes the check for previously-saved exit statuses unconditional. Let'

Re: AIX and Interix also do early PID recycling.

2012-07-26 Thread Chet Ramey
On 7/25/12 1:08 PM, Chet Ramey wrote: > On 7/25/12 11:33 AM, Andreas Schwab wrote: > >> I cannot see how CHILD_MAX is related to pid reuse. CHILD_MAX is a >> per-user limit, but the pid namespace is global. If the shell forks a >> new process, and the pid of it matches one of the previously used

Re: AIX and Interix also do early PID recycling.

2012-07-26 Thread Michael Haubenwallner
On 07/25/12 19:06, Chet Ramey wrote: Well, _SC_CHILD_MAX is documented across platforms as: Heck, even POSIX specifies CHILD_MAX as: "Maximum number of simultaneous processes per real user ID." Also, one Linux machine actually shows the _SC_CHILD_MAX value equal to kernel.pid_max (32768 here

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 10:57 AM, Roman Rakus wrote: > And there is also a problem when you change the `ulimit -u' value. > See https://bugzilla.redhat.com/show_bug.cgi?id=832997 It's not clear to me that it's desirable to attempt to remember a potentially unlimited number of process exit statuses. You have

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 11:33 AM, Andreas Schwab wrote: > I cannot see how CHILD_MAX is related to pid reuse. CHILD_MAX is a > per-user limit, but the pid namespace is global. If the shell forks a > new process, and the pid of it matches one of the previously used pids > for asynchronous jobs it can surely d

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 11:59 AM, Michael Haubenwallner wrote: > > On 07/25/2012 04:50 PM, Chet Ramey wrote: >>> The AIX 6.1 I've debugged on has: >>> #define CHILD_MAX 128 >>> #define _POSIX_CHILD_MAX 25 >>> sysconf(_SC_CHILD_MAX) = 1024 > >> Bash prefers sysconf(_SC_CHILD_MAX) and will use it over the

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Michael Haubenwallner
On 07/25/2012 04:50 PM, Chet Ramey wrote: >> The AIX 6.1 I've debugged on has: >> #define CHILD_MAX 128 >> #define _POSIX_CHILD_MAX 25 >> sysconf(_SC_CHILD_MAX) = 1024 > Bash prefers sysconf(_SC_CHILD_MAX) and will use it over the other > defines (lib/sh/oslib.c:getmaxchild()). I don't kno

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Andreas Schwab
Chet Ramey writes: > And that's the problem. Bash assumes that there's a PID space at least as > large as CHILD_MAX, and that the kernel will use all of it before reusing > any PID in the space. Posix says that shells must remember up to CHILD_MAX > statuses of terminated asynchronous children

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 10:11 AM, Michael Haubenwallner wrote: > > On 07/25/2012 02:14 PM, Greg Wooledge wrote: >> On Wed, Jul 25, 2012 at 09:59:28AM +0200, Michael Haubenwallner wrote: >>> OTOH, AFAICT, as long as a PID isn't waitpid()ed for, it isn't reused by >>> fork(). >>> However, I'm unable to find tha

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Roman Rakus
On 07/25/2012 04:52 PM, Michael Haubenwallner wrote: Got it: The value used for js.c_childmax isn't 128, but 1024. In lib/sh/oslib.c, getmaxchild() prefers sysconf(_SC_CHILD_MAX) over CHILD_MAX over MAXUPRC. But sysconf(_SC_CHILD_MAX) does return the number of "processes per real user id" (si

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 9:44 AM, Steven W. Orr wrote: > One thing that just springs to mind: Under AIX, I believe (but don't take > this as gospel) that PIDs are not strictly ascending. The low byte may > increment, but it might be another byte in the pid_t that gets the > increment, and the byte that incremen

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 9:20 AM, Michael Haubenwallner wrote: > > On 07/25/2012 09:59 AM, Michael Haubenwallner wrote: >> On 07/25/2012 03:05 AM, Chet Ramey wrote: >>> Bash holds on to the status of all terminated processes, not just >>> background ones, and only checks for the presence of a newly-forked PID >

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Michael Haubenwallner
On 07/25/2012 03:20 PM, Michael Haubenwallner wrote: > On 07/25/2012 09:59 AM, Michael Haubenwallner wrote: >> On 07/25/2012 03:05 AM, Chet Ramey wrote: >>> Bash holds on to the status of all terminated processes, not just >>> background ones, and only checks for the presence of a newly-forked PID

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 8:14 AM, Greg Wooledge wrote: > On Wed, Jul 25, 2012 at 09:59:28AM +0200, Michael Haubenwallner wrote: >> OTOH, AFAICT, as long as a PID isn't waitpid()ed for, it isn't reused by >> fork(). >> However, I'm unable to find that in the POSIX spec. > > A process that hasn't been waited for

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Chet Ramey
On 7/25/12 3:59 AM, Michael Haubenwallner wrote: > > On 07/25/2012 03:05 AM, Chet Ramey wrote: >> Bash assumes that there's a PID space at least as >> large as CHILD_MAX, and that the kernel will use all of it before reusing >> any PID in the space. Posix says that shells must remember up to CHIL

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Michael Haubenwallner
On 07/25/2012 02:14 PM, Greg Wooledge wrote: > On Wed, Jul 25, 2012 at 09:59:28AM +0200, Michael Haubenwallner wrote: >> OTOH, AFAICT, as long as a PID isn't waitpid()ed for, it isn't reused by >> fork(). >> However, I'm unable to find that in the POSIX spec. > > A process that hasn't been waite

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Steven W. Orr
On 7/25/2012 9:20 AM, Michael Haubenwallner wrote: On 07/25/2012 09:59 AM, Michael Haubenwallner wrote: On 07/25/2012 03:05 AM, Chet Ramey wrote: Bash holds on to the status of all terminated processes, not just background ones, and only checks for the presence of a newly-forked PID in that li

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Michael Haubenwallner
On 07/25/2012 09:59 AM, Michael Haubenwallner wrote: > On 07/25/2012 03:05 AM, Chet Ramey wrote: >> Bash holds on to the status of all terminated processes, not just >> background ones, and only checks for the presence of a newly-forked PID >> in that list if the list size exceeds CHILD_MAX. > Th

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Steven W. Orr
On 7/25/2012 8:14 AM, Greg Wooledge wrote: On Wed, Jul 25, 2012 at 09:59:28AM +0200, Michael Haubenwallner wrote: OTOH, AFAICT, as long as a PID isn't waitpid()ed for, it isn't reused by fork(). However, I'm unable to find that in the POSIX spec. A process that hasn't been waited for should be

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Greg Wooledge
On Wed, Jul 25, 2012 at 09:59:28AM +0200, Michael Haubenwallner wrote: > OTOH, AFAICT, as long as a PID isn't waitpid()ed for, it isn't reused by > fork(). > However, I'm unable to find that in the POSIX spec. A process that hasn't been waited for should become a zombie, which should be sufficien

Re: AIX and Interix also do early PID recycling.

2012-07-25 Thread Michael Haubenwallner
On 07/25/2012 03:05 AM, Chet Ramey wrote: > Bash assumes that there's a PID space at least as > large as CHILD_MAX, and that the kernel will use all of it before reusing > any PID in the space. Posix says that shells must remember up to CHILD_MAX > statuses of terminated asynchronous children (th

Re: AIX and Interix also do early PID recycling.

2012-07-24 Thread Chet Ramey
On 7/24/12 12:54 PM, Michael Haubenwallner wrote: >> (What >> would you expect bash to do about it in the first place?) It may also >> interest you to know that there are some operating systems that use >> random PID allocation, instead of sequential (OpenBSD for example). > > PID randomisation

Re: AIX and Interix also do early PID recycling.

2012-07-24 Thread Michael Haubenwallner
On 07/24/2012 05:49 PM, Greg Wooledge wrote: > On Tue, Jul 24, 2012 at 05:03:36PM +0200, michael.haubenwall...@salomon.at > wrote: >> Description: >> On AIX (5.3, 6.1, 7.1), as well as on Interix (any version) I do >> encounter >> some race condition in a code similar to: >> i

Re: AIX and Interix also do early PID recycling.

2012-07-24 Thread Greg Wooledge
On Tue, Jul 24, 2012 at 05:03:36PM +0200, michael.haubenwall...@salomon.at wrote: > Description: > On AIX (5.3, 6.1, 7.1), as well as on Interix (any version) I do > encounter > some race condition in a code similar to: > if grep "unwanted" /some/nonexistent/filename >

AIX and Interix also do early PID recycling.

2012-07-24 Thread michael . haubenwallner
Configuration Information [Automatically generated, do not change]: Machine: powerpc OS: aix5.3.0.0 Compiler: powerpc-ibm-aix5.3.0.0-gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='powerpc' -DCONF_OSTYPE='aix5.3.0.0' -DCONF_MACHTYPE='powerpc-ibm-aix5.3.0.0' -DCONF_VENDOR='ibm' -DLOCALE