On Wed, Dec 11, 2013 at 12:28 AM, Ben Reser <ben@> wrote: > I came to the conclusion that there was an LDAP problem as well. > > Things that would be interesting to know is if the httpd version changed > along > with the Subversion version or if httpd was left alone and only Subversion > was > updated. >
We have experienced the 100% CPU consumption for some time now as well. We are using mpm_winnit and LDAP. I did manage to get a stack trace of the thread (085:a1c) running in a hard loop consuming 100% of 1 CPU on a 4 CPU machine. Here is our version info and the trace: [Thu Apr 17 10:27:28.528141 2014] [mpm_winnt:notice] [pid 3844:tid 436] AH00455: Apache/2.4.7 (Win64) SVN/1.8.8 OpenSSL/1.0.1f configured -- resuming normal operations [Thu Apr 17 10:27:28.528141 2014] [mpm_winnt:notice] [pid 3844:tid 436] AH00456: Server built: Feb 14 2014 06:04:26 085:a1c libaprutil_1!apr_reslist_cleanup_order_set+0xc2 libaprutil_1!apr_rmm_calloc+0x84 mod_ldap+0x61d9 mod_ldap+0x6907 mod_ldap+0x39f2 mod_authnz_ldap+0x1ae3 mod_auth_basic+0x17f7 libhttpd!ap_run_check_user_id+0x35 libhttpd!ap_process_request_internal+0x3c4 libhttpd!ap_die+0x4bf libhttpd!ap_die+0x577 libhttpd!ap_psignature+0x1865 libhttpd!ap_run_process_connection+0x35 libhttpd!ap_regkey_value_remove+0x1603 kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d 4 other threads had similar stack traces and appeared to be looping as well though those threads appeared to be running between 30% to 50%. In aggregate all of the threads of the httpd child process consumed 99% to 100% of our 4 CPU server. My conjecture is that there is some thread safety issue (please excuse my lack of proper thread terminology) that is causing the looping when multiple threads enter the same LDAP code shown in the traces. Once that happens the thread that passes connections into the worker threads can't do and so that thread eventually uses up all the worker threads but is unable to actually pass the work off to the workers. Most of the worker threads stack traces show them blocked on "ntdll!ZwRemoveIoCompletion+0xa" which I think is the normal wait state. Here's a two of the other four suspect stack traces which are all similar. 087:1264 libaprutil_1!apr_reslist_cleanup_order_set+0xc2 libaprutil_1!apr_rmm_calloc+0x84 mod_ldap+0x624b mod_ldap+0x5fc6 mod_ldap+0x69c1 mod_ldap+0x39f2 mod_authnz_ldap+0x1ae3 mod_auth_basic+0x17f7 libhttpd!ap_run_check_user_id+0x35 libhttpd!ap_process_request_internal+0x3c4 libhttpd!ap_die+0x4bf libhttpd!ap_die+0x577 libhttpd!ap_psignature+0x1865 libhttpd!ap_run_process_connection+0x35 libhttpd!ap_regkey_value_remove+0x1603 kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d 090:7b0 libaprutil_1!apr_reslist_cleanup_order_set+0xc6 libaprutil_1!apr_rmm_calloc+0x84 mod_ldap+0x61d9 mod_ldap+0x6907 mod_ldap+0x39f2 mod_authnz_ldap+0x1ae3 mod_auth_basic+0x17f7 libhttpd!ap_run_check_user_id+0x35 libhttpd!ap_process_request_internal+0x3c4 libhttpd!ap_die+0x4bf libhttpd!ap_die+0x577 libhttpd!ap_psignature+0x1865 libhttpd!ap_run_process_connection+0x35 libhttpd!ap_regkey_value_remove+0x1603 kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d We upgraded to the latest version of Subversion on Monday and the problem reoccurred early this morning though we were not able to get a process dump. [Wed Apr 23 04:15:17.065606 2014] [mpm_winnt:notice] [pid 1728:tid 436] AH00455: Apache/2.4.9 (Win64) SVN/1.8.8 OpenSSL/1.0.1g configured -- resuming normal operations [Wed Apr 23 04:15:17.065606 2014] [mpm_winnt:notice] [pid 1728:tid 436] AH00456: Server built: Apr 8 2014 03:06:16 Any advice or suggestions for alleviating this chronic issue would be appreciated. Let me know if anyone wants additional information or clarification. mark w -- View this message in context: http://subversion.1072662.n5.nabble.com/Subversion-1-8-httpd-exe-taking-100-CPU-tp182657p188345.html Sent from the Subversion Users mailing list archive at Nabble.com.
