Slow signal delivery to server process with heavy I/O
Hi All, I've noticed that asynchronous signals such as SIGINT, SIGTERM etc are delivered to my process long after the signal is sent if the receiving process is handling lots of I/O. My process is a multi-threaded web server. It's got one thread waiting on 'select' to accept incoming connections and a thread pool which reads the data with 'recv'. When I batter the web server with incoming traffic and I try to shutdown the server by sending a SIGINT or SIGTERM, I have observed that the web server finishes handling the incoming traffic before the kernel dispatches the signal to the process. It appears that the 'select' and 'recv' calls are getting highest priority with regard to scheduling. I realize this test may appear unnatural and is perhaps unrealistic, but I would like to be able to shutdown my server gracefully within a reasonable amount of time, no matter what kind of load it is handling. Don't want to have to wait several minutes for my signals to get handled under heavy load. Could someone please explain why signal delivery is slow under these conditions? Thanks in advance, Dallas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktilggiesk7yggz-ujpje3epm8el5h8ormh4xs...@mail.gmail.com
segfault error 4 in libpthread-2.13.so X86_64
Hi All, I've got a multi-threaded C++ program that has been working for years that is segfaulting in the pthread library at exit after upgrading to squeeze. It litters my kernel log with the following message: segfault at 67eaf9d0 ip 7f4568e87d7c sp 7fff4571fe00 error 4 in libpthread-2.13.so[7f4568e8+17000 I can produce a core file, but unfortunately I can't see the symbols inside the pthread lib to determine where it is crashing. Is there a pthread library source package or debug version of libc / pthreads that I can try? # uname -a Linux bertha 2.6.32-5-amd64 #1 SMP Mon Oct 3 03:59:20 UTC 2011 x86_64 GNU/Linux Thanks, Dallas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAE9DZUTJebtQCRVWza9DzH5h0OKKVhRoZWVY-i=ovc--ecc...@mail.gmail.com
Re: segfault error 4 in libpthread-2.13.so X86_64
How can I link with libc6-dbg instead of regular libc6? Thanks. On Sat, Oct 29, 2011 at 2:48 PM, Chow Loong Jin wrote: > On 30/10/2011 03:34, Dallas Clement wrote: >> >> I can produce a core file, but unfortunately I can't see the symbols >> inside the pthread lib to determine where it is crashing. Is there a >> pthread library source package or debug version of libc / pthreads >> that I can try? > > Debugging symbols can be found in the libc6-dbg package. > > -- > Kind regards, > Loong Jin > > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAE9DZUT-cAxZYhE00jQxE_a-E5sStd9f+uyVA=g0frzhatl...@mail.gmail.com
Re: segfault error 4 in libpthread-2.13.so X86_64
I can see that I've got a debug version of the pthreads library installed on my system. Just don't know how to link with it vs the non-debug version. /usr/lib/debug/lib/x86_64-linux-gnu# ls libpthread-2.13.so On Sat, Oct 29, 2011 at 4:20 PM, Bernd Zeimetz wrote: > Do not use TOFU on Debian lists. It just makes a mess. > > On 10/29/2011 09:55 PM, Dallas Clement wrote: >> How can I link with libc6-dbg instead of regular libc6? Thanks. > > http://wiki.debian.org/HowToGetABacktrace > > -- > Bernd Zeimetz Debian GNU/Linux Developer > http://bzed.de http://www.debian.org > GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F > > > -- > To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: http://lists.debian.org/4eac6e04.1080...@bzed.de > > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cae9dzusvgdk131xii9xj0bgdv7mkw8akwnbh+aq_3jgo25q...@mail.gmail.com
Re: segfault error 4 in libpthread-2.13.so X86_64
That certainly makes sense. I'm still not seeing the symbols though. My kernel log show that the segfault came from the pthreads library, yet when I load the core file into gdb, I'm not seeing any symbols. (gdb) bt #0 0x7f04e3102d7c in ?? () #1 0x7f04e3e73a60 in ?? () #2 0x7f04e47db695 in ?? () #3 0xe212a700 in ?? () #4 0x7f04e3e73a60 in ?? () #5 0x in ?? () Got any other ideas for me to try? On Sat, Oct 29, 2011 at 4:43 PM, brian m. carlson wrote: > On Sat, Oct 29, 2011 at 02:55:54PM -0500, Dallas Clement wrote: >> How can I link with libc6-dbg instead of regular libc6? Thanks. > > You don't. The files in /usr/lib/debug contain only debugging symbols > that have been stripped from the normal binary. You simply link against > libc6 as normal and gdb (and other programs) will find the debugging > symbols automatically. > > -- > brian m. carlson / brian with sandals: Houston, Texas, US > +1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only > OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187 > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cae9dzut_s0tudyvfoht6ahek3gruq5jnfgnulbtphdg3-du...@mail.gmail.com
Re: segfault error 4 in libpthread-2.13.so X86_64
The symbols are definitely present on my system. When I step through the execution, I can see where it is crashing inside the pthread library. The last thing it does is join with one thread that was previously spawned. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f650debe720 (LWP 3784)] 0x7f650c5f7d7c in pthread_join (threadid=190969600, thread_return=0x0) at pthread_join.c:46 On Sun, Oct 30, 2011 at 1:56 AM, Chow Loong Jin wrote: > On 30/10/2011 06:22, Dallas Clement wrote: >> That certainly makes sense. I'm still not seeing the symbols though. >> My kernel log show that the segfault came from the pthreads library, >> yet when I load the core file into gdb, I'm not seeing any symbols. >> >> (gdb) bt >> #0 0x7f04e3102d7c in ?? () >> #1 0x7f04e3e73a60 in ?? () >> #2 0x7f04e47db695 in ?? () >> #3 0xe212a700 in ?? () >> #4 0x7f04e3e73a60 in ?? () >> #5 0x in ?? () >> >> Got any other ideas for me to try? > > You need to load the core file, *and* the program. Something like... > $ gdb -c $corefile $path_to_program > > -- > Kind regards, > Loong Jin > > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cae9dzuq6wsjrsqrh5yrq0m_fadsjlkuzzqeemd_ydspok4o...@mail.gmail.com
Re: segfault error 4 in libpthread-2.13.so X86_64
Just pointing out that I'm seeing a segfault inside pthread_join of libpthread-2.13.so. I need to look at the pthread source before I can tell why it crashed. I will also try to roll back to an older version of this library to confirm that I was not seeing this crash earlier. Will report back what I find. On Mon, Oct 31, 2011 at 1:53 AM, Chow Loong Jin wrote: > On 31/10/2011 13:13, Dallas Clement wrote: >> The symbols are definitely present on my system. When I step through >> the execution, I can see where it is crashing inside the pthread >> library. The last thing it does is join with one thread that was >> previously spawned. > > And so? > > -- > Kind regards, > Loong Jin > > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAE9DZURoqEbsc4Xf1vA78T=n7be5mxdbyspugrsbp2fvtz8...@mail.gmail.com
