hi, Looking at the bottom of this stack trace:
J java.lang.ref.Finalizer.invokeFinalizeMethod(Ljava/lang/Object;)V J java.lang.ref.Finalizer.runFinalizer()V J java.lang.ref.Finalizer$FinalizerThread.run()V Called by the garbage collector on an object when garbage collection determines that there are no more references to the object. j com.sun.net.ssl.internal.ssl.SSLSocketImpl.close()V+44 j com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.finalize()V+1 Java's JSSE com.sun.net.ssl.internal.ssl finalize method is being called J org.mozilla.jss.provider.java.security.JSSMessageDigestSpi.engineUpdate ([BII)V J java.security.MessageDigest$Delegate.engineUpdate([BII)V J java.security.MessageDigest.update([B)VThe stack trace then goes on to show that the Mozilla-JSS JCE provider is being called by Java's JSSE SSLSocket instance to compute an HMAC on message that will inform the other end that this socket is closing. Then finally crashes in NSPR with an invalid argument...
Are you using JSS for SSL, or Java's JSSE? Have you monitored your server memory usage, is the process running out of memory? If you're doing JSS for SSL, and are creating any instances of JSSE SSL sockets then this issue
is likely stack corruption.can you create a bug on JSS, and try to attach a reproducible test case, or at least in the bug detail description of what your server is doing, how it is configured, and what load it takes to crash.
-glen ksreedha...@gmail.com wrote:
Hi Wan-Teh, Before I try to get the stack track using gdb on the core file, as I made it to crash with your suggested change, Java crashed with this error. Actually it also stack trace. Can you see anything from it. I appreciate your help. This is a must fix for us. # # An unexpected error has been detected by Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x712a97e2, pid=10873, tid=1904720784 # # Java VM: Java HotSpot(TM) Client VM (1.6.0_01-b06 mixed mode) # Problematic frame: # C [libnspr4.so+0x277e2] PR_Lock+0x6b # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x080ae000): JavaThread "Finalizer" daemon [_thread_in_native, id=10878] siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000000 Registers: EAX=0x00000000, EBX=0x712be460, ECX=0x7187a6a4, EDX=0x00000000 ESP=0x7187a748, EBP=0x7187a770, ESI=0x08378368, EDI=0x080ae0e8 EIP=0x712a97e2, CR2=0x00000000, EFLAGS=0x00010206 Top of Stack: (sp=0x7187a748) 0x7187a748: 712bc3dc 00000016 00000014 00000014 0x7187a758: 00000000 74568ba0 7187a780 00000016 0x7187a768: 00000000 713d3608 7187a790 7135c4aa 0x7187a778: 089757d8 00000000 7187a7c0 7135c480 0x7187a788: 08378368 713d3608 7187a7c0 7135d8b9 0x7187a798: 09581cd8 080ae0e8 7187a7c0 7135d86d 0x7187a7a8: 080ae0e8 7187a810 00000000 00000000 0x7187a7b8: 713fbf1c 08378368 7187a7f0 713e6203 Instructions: (pc=0x712a97e2) 0x712a97d2: 83 7c df ff ff 89 04 24 e8 41 07 fe ff 8b 45 f8 0x712a97e2: c7 00 00 00 00 00 83 7d f4 00 74 20 c7 44 24 08 Stack: [0x7185b000,0x7187c000), sp=0x7187a748, free space=125k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [libnspr4.so+0x277e2] PR_Lock+0x6b C [libnss3.so+0x2a4aa] C [libnss3.so+0x2b8b9] PK11_DigestOp+0x59 C [libjss4.so+0x11203] Java_org_mozilla_jss_pkcs11_PK11MessageDigest_update+0x5f J org.mozilla.jss.pkcs11.PK11MessageDigest.update(Lorg/mozilla/jss/ pkcs11/CipherContextProxy;[BII)V J org.mozilla.jss.pkcs11.PK11MessageDigest.update([BII)V J org.mozilla.jss.provider.java.security.JSSMessageDigestSpi.engineUpdate ([BII)V J java.security.MessageDigest$Delegate.engineUpdate([BII)V J java.security.MessageDigest.update([B)V j com.sun.crypto.provider.SslMacCore.update([BII)V+16 j com.sun.crypto.provider.SslMacCore$SslMacSHA1.engineUpdate([BII)V+7 j javax.crypto.Mac.update([B)V+33 j com.sun.net.ssl.internal.ssl.MAC.compute(BLjava/nio/ByteBuffer;[BII) [B+60 j com.sun.net.ssl.internal.ssl.MAC.compute(B[BII)[B+7 j com.sun.net.ssl.internal.ssl.OutputRecord.addMAC(Lcom/sun/net/ssl/ internal/ssl/MAC;)V+36 J com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Lcom/sun/net/ ssl/internal/ssl/OutputRecord;)V j com.sun.net.ssl.internal.ssl.SSLSocketImpl.sendAlert(BB)V+216 j com.sun.net.ssl.internal.ssl.SSLSocketImpl.warning(B)V+3 j com.sun.net.ssl.internal.ssl.SSLSocketImpl.closeInternal(Z)V+192 j com.sun.net.ssl.internal.ssl.SSLSocketImpl.close()V+44 j com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.finalize()V+1 v ~StubRoutines::call_stub V [libjvm.so+0x209a4d] V [libjvm.so+0x305bc8] V [libjvm.so+0x2098e0] V [libjvm.so+0x232f77] V [libjvm.so+0x2198ad] C [libjava.so+0xcb9e] Java_java_lang_ref_Finalizer_invokeFinalizeMethod+0x6e J java.lang.ref.Finalizer.invokeFinalizeMethod(Ljava/lang/Object;)V J java.lang.ref.Finalizer.runFinalizer()V J java.lang.ref.Finalizer$FinalizerThread.run()V v ~StubRoutines::call_stub V [libjvm.so+0x209a4d] V [libjvm.so+0x305bc8] V [libjvm.so+0x209360] V [libjvm.so+0x2093ed] V [libjvm.so+0x279605] V [libjvm.so+0x38076f] V [libjvm.so+0x306aa3] C [libpthread.so.0+0x545b] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J org.mozilla.jss.pkcs11.PK11MessageDigest.update(Lorg/mozilla/jss/ pkcs11/CipherContextProxy;[BII)V J org.mozilla.jss.pkcs11.PK11MessageDigest.update([BII)V J org.mozilla.jss.provider.java.security.JSSMessageDigestSpi.engineUpdate ([BII)V J java.security.MessageDigest$Delegate.engineUpdate([BII)V J java.security.MessageDigest.update([B)V j com.sun.crypto.provider.SslMacCore.update([BII)V+16 j com.sun.crypto.provider.SslMacCore$SslMacSHA1.engineUpdate([BII)V+7 j javax.crypto.Mac.update([B)V+33 j com.sun.net.ssl.internal.ssl.MAC.compute(BLjava/nio/ByteBuffer;[BII) [B+60 j com.sun.net.ssl.internal.ssl.MAC.compute(B[BII)[B+7 j com.sun.net.ssl.internal.ssl.OutputRecord.addMAC(Lcom/sun/net/ssl/ internal/ssl/MAC;)V+36 J com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Lcom/sun/net/ ssl/internal/ssl/OutputRecord;)V j com.sun.net.ssl.internal.ssl.SSLSocketImpl.sendAlert(BB)V+216 j com.sun.net.ssl.internal.ssl.SSLSocketImpl.warning(B)V+3 j com.sun.net.ssl.internal.ssl.SSLSocketImpl.closeInternal(Z)V+192 j com.sun.net.ssl.internal.ssl.SSLSocketImpl.close()V+44 j com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.finalize()V+1 v ~StubRoutines::call_stub J java.lang.ref.Finalizer.invokeFinalizeMethod(Ljava/lang/Object;)V J java.lang.ref.Finalizer.runFinalizer()V J java.lang.ref.Finalizer$FinalizerThread.run()V v ~StubRoutines::call_stub Thanks, Sreedhar On May 8, 2:04 pm, Wan-Teh Chang <w...@google.com> wrote:On Fri, May 8, 2009 at 12:58 PM, <ksreedha...@gmail.com> wrote:Hi Wan-Teh,Before the crash, I get the output of "pthread_mutex_lock fails with22".So, rv value is 22. Can you figure out the something out of this.I remember you're using CentOS Linux, right? On Linux, errno 22 is EINVAL (invalid argument). The pthread_mutex_lock man page lists two reasons for pthread_mutex_lock to fail with EINVAL:http://www.opengroup.org/onlinepubs/007908775/xsh/pthread_mutex_unloc... The more likely reason is: The value specified by mutex does not refer to an initialised mutex object. So I guess the PRLock in question is not initialized or has been destroyed. Could you try to get a stack trace? You can cause NSPR to crash: rv = pthread_mutex_lock(&lock->mutex); if (rv != 0) { /* ADD THIS */ int *p = NULL; printf("pthread_mutex_lock fails with %d\n", rv); *p = 0; /* THIS WILL CRASH */ } PR_ASSERT(0 == rv); Make sure your Java Server dumps core when it crashes, and use gdb on the core file to get the stack trace. Alternatively, you can just run your Java Server in gdb and let it crash. Thanks, Wan-Teh
smime.p7s
Description: S/MIME Cryptographic Signature
-- dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
