Hi, (gdb) thread apply all bt
Thread 3 (Thread 0x7ff0094ed700 (LWP 31498)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007ff017af0ecd in xmltooling::ReloadableXMLFile::reload_fn(void*) () from /usr/lib/x86_64-linux-gnu/libxmltooling-lite.so.6 #2 0x00007ff01f7ea064 in start_thread (arg=0x7ff0094ed700) at pthread_create.c:309 #3 0x00007ff01f51f62d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 2 (Thread 0x7ff000cab700 (LWP 31510)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x00007ff017afbec3 in xmltooling::CondWaitImpl::timedwait(xmltooling::Mutex*, int) () from /usr/lib/x86_64-linux-gnu/libxmltooling-lite.so.6 #2 0x00007ff0177d9241 in ?? () from /usr/lib/x86_64-linux-gnu/libshibsp-lite.so.6 #3 0x00007ff01f7ea064 in start_thread (arg=0x7ff000cab700) at pthread_create.c:309 #4 0x00007ff01f51f62d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 Thread 1 (Thread 0x7ff020579780 (LWP 31497)): #0 0x00007ff01f7f1489 in __libc_waitpid (pid=31518, stat_loc=0x7ffc44121764, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40 #1 0x00007ff01c751b06 in itk_fork_process () from /usr/lib/apache2/modules/mpm_itk.so #2 0x00007ff020351f40 in ap_run_process_connection () #3 0x00007ff01c13f7ba in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #4 0x00007ff01c13fa01 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #5 0x00007ff01c140667 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #6 0x00007ff02032c7ee in ap_run_mpm () #7 0x00007ff0203255f3 in main () (gdb)
Do you mean that disabling logging fixed the problem for you and there are no hung workers in that case?
Yes, I commented out all lines in /etc/shibboleth/native.logger and it solve my problem.
Also, please provide the output of the "thread apply all bt" gdb command on a hung process. This looks like a log4shib threading problem, probably inherited from log4cpp. -- Thanks, Feri
Thank you Pavel Polacek