Hi All, We are using Mozilla LDAP C SDK 5.12 in our application. We are currently switching from synchronous calls to asynchronous calls, so that even when Directory Server (DS) hangs our servers do not hang.
To simulate DS hang, we hooked up a small plugin (delete pre-operation) into iPlanet DS. Basically this plugin contains one infinite loop. So, when we do any operation in our servers, which internally involves some entry deletion from DS, DS is getting hanged. [ Please note that, the number of operation threads in DS is 30. With our delete request, one operation thread might be falling in infinite loop. But, some how it seems like total DS / all operation threads are getting blocked, because even if we try to connect to DS with Ldp, connection is not getting established. ] Our server (one dedicated thread) periodically tries to estabblish a connection with DS, to check its healthiness. When DS hangs because of some delete operation request, the above thread that tries to establish a connection with DS is getting blocked in ldap_simple_bind asynchronous method, which is not as supposed as we are using asynchronous API. Key point to note here is that, we are observing this behavior only when connection type between our server and DS is configured as SSL. In Open mode, the above thread is not getting blocked in ldap_simple_bind, when DS is in hang state. Stack trace of blocking thread is as follows: # ChildEBP RetAddr Args to Child 00 03ac83bc 7c59a072 00000304 00000000 00000000 ntdll!ZwWaitForSingleObject+0xb (FPO: [3,0,0]) 01 03ac83e4 7c57b3e9 00000304 ffffffff 00000000 KERNEL32!WaitForSingleObjectEx+0x71 (FPO: [Non-Fpo]) 02 03ac83f4 30036d02 00000304 ffffffff 0137d9de KERNEL32!WaitForSingleObject+0xf (FPO: [2,0,0]) 03 03ac841c 30038399 0229ae50 ffffffff 00000001 libobnspr4!_PR_MD_WAIT(struct PRThread * thread = 0x0229ae50, unsigned int ticks = 0xffffffff)+0x72 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\nsprpub\pr\src\md\windows\ntio.c @ 528] 04 03ac8430 30039c71 0229ae50 ffffffff 0229ae50 libobnspr4!_NT_IO_WAIT(struct PRThread * thread = 0x0229ae50, unsigned int timeout = 0xffffffff)+0xe9 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\nsprpub\pr\src\md\windows\ntio.c @ 743] 05 03ac8454 30026cac 04a70138 04b2377c 00000005 libobnspr4!_PR_MD_RECV(struct PRFileDesc * fd = 0x04a70138, void * buf = 0x04b2377c, int amount = 5, int flags = 0, unsigned int timeout = 0xffffffff)+0x2f1 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\nsprpub\pr\src\md\windows\ntio.c @ 1756] 06 03ac847c 30008286 04a70138 04b2377c 00000005 libobnspr4!SocketRecv(struct PRFileDesc * fd = 0x04a70138, void * buf = 0x04b2377c, int amount = 5, int flags = 0, unsigned int timeout = 0xffffffff)+0x27c (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\nsprpub\pr\src\io\prsocket.c @ 638] 07 03ac8498 0137d935 04b23a70 04b2377c 00000005 libobnspr4!pl_DefRecv(struct PRFileDesc * fd = 0x04b23a70, void * buf = 0x04b2377c, int amount = 5, int flags = 0, unsigned int timeout = 0xffffffff)+0x66 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\nsprpub\pr\src\io\prlayer.c @ 277] 08 03ac84c0 01374e70 04b23598 04b2377c 00000005 obssl3!ssl_DefRecv(struct sslSocketStr * ss = 0x04b23598, unsigned char * buf = 0x04b2377c "", int len = 5, int flags = 0)+0x33 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\ssldef.c @ 91] 09 03ac84f4 01374cad 04b23598 04b2373c 00000000 obssl3!ssl3_GatherData(struct sslSocketStr * ss = 0x04b23598, struct sslGatherStr * gs = 0x04b2373c, int flags = 0)+0xf6 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\ssl3gthr.c @ 87] 0a 03ac8518 01375fb5 04b23598 00000000 04b23598 obssl3!ssl3_GatherCompleteHandshake(struct sslSocketStr * ss = 0x04b23598, int flags = 0)+0x4d (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\ssl3gthr.c @ 192] 0b 03ac852c 0138008f 04b23598 00000000 00000001 obssl3!ssl_GatherRecord1stHandshake(struct sslSocketStr * ss = 0x04b23598)+0x62 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\sslcon.c @ 1256] 0c 03ac8540 013818e2 04b23598 00000000 03ac8570 obssl3!ssl_Do1stHandshake(struct sslSocketStr * ss = 0x04b23598)+0x1ef (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\sslsecur.c @ 145] 0d 03ac8550 0138730b 04b23598 04b18504 0000002c obssl3!ssl_SecureSend(struct sslSocketStr * ss = 0x04b23598, unsigned char * buf = 0x04b18504 "0*???", int len = 44, int flags = 0)+0x184 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\sslsecur.c @ 1024] 0e 03ac8570 300063bf 04b14928 04b18504 0000002c obssl3!ssl_Send(struct PRFileDesc * fd = 0x04b14928, void * buf = 0x04b18504, int len = 44, int flags = 0, unsigned int timeout = 0xffffffff)+0xa5 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\security\nss\lib\ssl\sslsock.c @ 1222] 0f 03ac858c 00281562 04b14928 04b18504 0000002c libobnspr4!PR_Send(struct PRFileDesc * fd = 0x04b14928, void * buf = 0x04b18504, int amount = 44, int flags = 0, unsigned int timeout = 0xffffffff)+0x1f (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\nsprpub\pr\src\io\priometh.c @ 221] 10 03ac85ac 10030e45 00000001 04b18504 0000002c OBNSLDAPPR32V50!prldap_write(int s = 1, void * buf = 0x04b18504, int len = 44, struct lextiof_socket_private * socketarg = 0x04b18930)+0x32 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\libprldap\ldappr-io.c @ 210] 11 03ac8628 1001be5c 04a6ff60 04b183e0 00000000 OBNSLDAP32V50!ber_flush(struct sockbuf * sb = 0x04a6ff60, struct berelement * ber = 0x04b183e0, int freeit = 0)+0x205 (FPO: [Non-Fpo]) (CONV: stdcall) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\liblber\io.c @ 417] 12 03ac8650 1001b676 04a6fd80 04a6ff60 04b183e0 OBNSLDAP32V50!nsldapi_send_ber_message(struct ldap * ld = 0x04a6fd80, struct sockbuf * sb = 0x04a6ff60, struct berelement * ber = 0x04b183e0, int freeit = 0)+0x6c (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\libldap\request.c @ 399] 13 03ac8880 1001aabf 04a6fd80 04b183e0 00000001 OBNSLDAP32V50!nsldapi_send_server_request(struct ldap * ld = 0x04a6fd80, struct berelement * ber = 0x04b183e0, int msgid = 1, struct ldapreq * parentreq = 0x00000000, struct ldap_server * srvlist = 0x00000000, struct ldap_conn * lc = 0x04b281c0, char * bindreqdn = 0x04b23b78 "cn=Directory Manager", int bind = 0)+0xb86 (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\libldap\request.c @ 334] 14 03ac89b0 1002442b 04a6fd80 00000001 00000060 OBNSLDAP32V50!nsldapi_send_initial_request(struct ldap * ld = 0x04a6fd80, int msgid = 1, unsigned long msgtype = 0x60, char * dn = 0x04b27ca8 "cn=Directory Manager", struct berelement * ber = 0x04b183e0)+0x6f (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\libldap\request.c @ 148] 15 03ac89e4 10023fdd 04a6fd80 04b27ca8 04b27cf0 OBNSLDAP32V50!simple_bind_nolock(struct ldap * ld = 0x04a6fd80, char * dn = 0x04b27ca8 "cn=Directory Manager", char * passwd = 0x04b27cf0 "oblixoblix", int unlock_permitted = 1)+0x41b (FPO: [Non-Fpo]) (CONV: cdecl) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\libldap\sbind.c @ 150] 16 03ac8b04 006cd518 04a6fd80 04b27ca8 04b27cf0 OBNSLDAP32V50!ldap_simple_bind(struct ldap * ld = 0x04a6fd80, char * dn = 0x04b27ca8 "cn=Directory Manager", char * passwd = 0x04b27cf0 "oblixoblix")+0x5d (FPO: [Non-Fpo]) (CONV: stdcall) [j:\sdk\mozilla\source\mozilla_new_home\ldapsdkcode\mozilla\directory\c-sdk\ldap\libraries\libldap\sbind.c @ 79] 17 03aca728 006ec1c1 04a6fd80 02292050 02292164 ois_server_d!LDAPSimpleBind(struct ldap * masterHandle = 0x04a6fd80, char * login = 0x02292050 "cn=Directory Manager", char * passwd = 0x02292164 "oblixoblix")+0x352 (FPO: [Non-Fpo]) (CONV: cdecl) [d:\p4\coreid\np_common\db\ldap\util\ldap_util3.cpp @ 504] 18 03acd2cc 00b3cef7 04a6fd80 02292050 02292164 ois_server_d!LDAPAsyncSimpleBind(struct ldap * ld = 0x04a6fd80, char * login = 0x02292050 "cn=Directory Manager", char * passwd = 0x02292164 "oblixoblix", struct ldapcontrol *** serverCtrls = 0x00000000, long timeout = -2)+0x13e (FPO: [Non-Fpo]) (CONV: cdecl) [d:\p4\coreid\np_common\db\ldap\util\ldap_util3.cpp @ 2968] 19 03ade454 005e301c 00000000 03adfb74 03ade878 ois_server_d!LDAPConnectionManager::_CreateAndOpenConnection(bool bSetup = false)+0x4b27 (FPO: [Non-Fpo]) (CONV: thiscall) [d:\p4\coreid\np_common\db\connection\ldap\ldap_connection_mngr.cpp @ 384] 1a 03ade870 005e729a 03adff78 cccccccc 0382ef80 ois_server_d!DBConnectionManager::PollLDAPServer(void)+0x6d (FPO: [Non-Fpo]) (CONV: thiscall) [d:\p4\coreid\np_common\db\utils\db_connection_mngr.cpp @ 482] 1b 03adfb84 005e6366 cccccccc cccccccc 0382ef80 ois_server_d!ConnectionWatcherThread::DoHeartBeat(void)+0x35a (FPO: [Non-Fpo]) (CONV: thiscall) [d:\p4\coreid\np_common\db\utils\db_failover.cpp @ 614] 1c 03adff88 0041919b cccccccc cccccccc 0382ef80 ois_server_d!ConnectionWatcherThread::Run(void)+0x232 (FPO: [Non-Fpo]) (CONV: thiscall) [d:\p4\coreid\np_common\db\utils\db_failover.cpp @ 406] 1d 03adffb4 7c57b396 0382ef80 cccccccc cccccccc ois_server_d!ObThread::ThreadFunc(void * arg = 0x0382ef80)+0x5d (FPO: [Non-Fpo]) (CONV: stdcall) [d:\p4\coreid\np_common\util\base\obthread.cpp @ 736] >From the above stack trace, it seems like Mozilla LDAP SDK is indefinitely waiting for some data from socket (SocketRecv method call). Can anybody infer what exactly this thread is doing inside? And why it is getting blocked in an asynchronous call? Thanks in advance, Ramakrishna. _______________________________________________ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
