Hi, I'm writing a server application using NSS. It appears to be working, but when it's time to call NSS_Shutdown(), it fails with SEC_ERROR_BUSY. I've been looking into it and it appears the client sockets leak PK11SlotInfo references. The leak only occurs when a handshake completes, and not if a client only connects and disconnects without doing anything. Here's what my program does:
I'm using NSS with a custom reactor (non-blocking I/O), so I've implemented a bottom NSPR I/O layer that calls my own socket functions. I register a layer ID once with PR_GetUniqueIdentity. I have a function which creates a PRFileDesc; it sets the methods table, the layer ID and the 'secret' pointer to a custom structure. I implement close, read, write, recv, send, poll, getpeername, getsocketoption and setsocketoption. Close frees everything that was allocated when the object was created. Write and read either succeed, fail with PR_WOULD_BLOCK_ERROR, or fail with PR_UNKNOWN_ERROR. Send and recv ignore the timeout argument and just call write and read. Poll simply sets out_flags to 0 and returns in_flags. Getsocketoption only recognizes PR_SockOpt_Nonblocking and sets the non_blocking output flag to true, and setsocketoption always fails. The main program first calls PR_Init, PR_GetUniqueIdentity, PK11_SetPasswordFunc, NSS_Init, NSS_SetDomesticPolicy, SSL_ConfigServerSessionIDCache, opens the certificate and key, then starts accepting connections (without using NSPR). When a client connects, it creates my PRFileDesc object from the system file descriptor, calls SSL_ImportFD, SSL_ResetHandshake and SSL_ConfigSecureServer. I/O is then attempted on the SSL socket, the handshake completes, and the secure connection works. When a client disconnects (PR_Read or PR_Write fails (except PR_WOULD_BLOCK_ERROR), or PR_Read returns 0), I call PR_Close on the SSL socket. The reactor polls the underlying sockets with help of the SSL socket's poll method. When a signal happens, I attempt to clean up everything. I PR_Close client SSL file descriptors, and go global cleanup with SECKEY_DestroyPrivateKey, CERT_DestroyCertificate, NSS_Shutdown and PR_Cleanup. What am I missing? Thank you, Ambroz -- dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
