Re: Lock order reversals in sys_pipe.c and kern_sig.c

[Brent Verner]

[2002-11-18 11:39] Alfred Perlstein said:
| * Kris Kennaway <[EMAIL PROTECTED]> [021118 11:06] wrote:
| > I've just turned witness back on on the bento cluster, and got the
| > following lock order reversals a number of times overnight:
| > 
| > Nov 18 07:45:40 <user.crit> gohan11 kernel: 1st 0xc6887200 pipe mutex (pipe mutex) 
|@ /local0/src-client/sys/kern/sys_pipe.c:465
| > Nov 18 07:45:40 <user.crit> gohan11 kernel: 2nd 0xc0447780 sigio lock (sigio lock) 
|@ /local0/src-client/sys/kern/kern_sig.c:2225
| > Nov 18 10:28:47 <user.crit> gohan10 kernel: 1st 0xc4941580 pipe mutex (pipe mutex) 
|@ /local0/src-client/sys/kern/sys_pipe.c:1038
| > [...]
| > 
| > Are these known problems?
| 
| Well now they are, I will investigate as time permits.

Maybe this will help.  The attached prog causes a LOR message.  I dug
thru the kernel source from fcntl to fsetown, but am little more than
lost at this point...

thanks.
  brent

-- 
"Develop your talent, man, and leave the world something. Records are 
really gifts from people. To think that an artist would love you enough
to share his music with anyone is a beautiful thing."  -- Duane Allman

#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>

/*

This program causes the following LOR.  Strangely, it will only cause
the LOR _once_.  To repeat, you must reboot.

lock order reversal
 1st 0xc526c180 pipe mutex (pipe mutex) @ /usr/src/sys/kern/sys_pipe.c:465
 2nd 0xc051ef80 sigio lock (sigio lock) @ /usr/src/sys/kern/kern_sig.c:2225
Debugger("witness_lock")
Stopped at      Debugger+0x54:  xchgl   %ebx,in_Debugger.0
db> trace
Debugger(c04961d5,c051ef80,c04ccf5a,c04ccf5a,c04cf7e1) at Debugger+0x54
witness_lock(c051ef80,8,c04cf7e1,8b1,c526c180) at witness_lock+0x667
_mtx_lock_flags(c051ef80,0,c04cf7e1,8b1,23) at _mtx_lock_flags+0xb1
pgsigio(c4f2fe58,17,0,1ad,0) at pgsigio+0x30
pipe_read(c4f28654,e0286c7c,c5024680,0,c4de28c0) at pipe_read+0x516
dofileread(c4de28c0,c4f28654,3,bfbffca3,1) at dofileread+0xd2
read(c4de28c0,e0286d10,c04e9042,407,3) at read+0x6b
syscall(2f,2f,2f,bfbffcd8,bfbffce0) at syscall+0x28e
Xint0x80_syscall() at Xint0x80_syscall+0x1d
--- syscall (3, FreeBSD ELF32, read), eip = 0x280b09c3, esp = 0xbfbffc7c, ebp =
0xbfbffcb0 ---

*/

int
main(int argc, char** argv)
{
  int fd[2];
  long flags;
  char buf[1];

  if (pipe(fd) == 0) {
    flags = fcntl(fd[0], F_GETFL, 0);
    fcntl(fd[0], F_SETFL, flags | O_NONBLOCK | O_ASYNC);
    fcntl(fd[0], F_SETOWN, getpid());/* <-- causes LOR in read() */
    read(fd[0], buf, 1);
    close(fd[0]);
    close(fd[1]);
  }
  else
  {
    perror("fifo");
  }
  return 0;
}