I assume that you're not using any Tomcat native libs? If you are, try removing them... if not (and the crash happened more than once in the same place) then it looks like a JVM bug rather than flakey hardware and the easiest path forward would be to try the latest Java6 (update 12).
-Yonik http://www.lucidimagination.com On Fri, Mar 20, 2009 at 12:42 PM, wojtekpia <wojte...@hotmail.com> wrote: > > I'm running Solr on Tomcat 6.0.18 with Java 6 update 7 on Windows 2003 64 > bit. Over the past month or so, my JVM has crashed twice with the error > below. Has anyone experienced this? My system is not heavily loaded, and the > crash seems to coincide with an update (via DIH). I'm running trunk code > from late January. Note that I update my index ~50 times per day, and this > crash has happened twice in the past month (so 2 of 1500 updates seem to > have triggered the crash). > > This Windows deployment is for demos, so I'm not too concerned about it. > Interestingly, my production deployment is on a 64 bit Linux system (same > versions of everything) and I haven't been able to reproduce the bug there. > > # > # An unexpected error has been detected by Java Runtime Environment: > # > # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000080e51c3, > pid=4404, tid=956 > # > # Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b23 mixed mode > windows-amd64) > # Problematic frame: > # V [jvm.dll+0xe51c3] > # > # If you would like to submit a bug report, please visit: > # http://java.sun.com/webapps/bugreport/crash.jsp > # > > --------------- T H R E A D --------------- > > Current thread (0x0000000001de2000): GCTaskThread [stack: > 0x0000000000000000,0x0000000000000000] [id=956] > > siginfo: ExceptionCode=0xc0000005, reading address 0xffffffffffffffff > > Registers: > EAX=0x3000000000000000, EBX=0x0000000001e40330, ECX=0x0000000184b49821, > EDX=0x0000000184b4b580 > ESP=0x0000000007cff9b0, EBP=0x0000000000000000, ESI=0x0000000184b4b580, > EDI=0x0000000000000935 > EIP=0x00000000080e51c3, EFLAGS=0x0000000000010206 > > Top of Stack: (sp=0x0000000007cff9b0) > 0x0000000007cff9b0: 0000000001e40330 0000000000000000 > 0x0000000007cff9c0: 0000000184b4dd88 0000000000000935 > 0x0000000007cff9d0: 0000000008464b08 0000000001dbbdc0 > 0x0000000007cff9e0: 0000000001dbf190 000000008a650000 > 0x0000000007cff9f0: 000000002f5b4000 00000002015f0000 > 0x0000000007cffa00: 0000000000020000 0000000001dbf2f0 > 0x0000000007cffa10: 0000000001e40330 0000000001dbf430 > 0x0000000007cffa20: 0000000001dbf4f0 0000000201602d18 > 0x0000000007cffa30: 0000000007effa00 0000000007cffb40 > 0x0000000007cffa40: 0000000000000000 0000000000000000 > 0x0000000007cffa50: 0000000000000000 000000000830484d > 0x0000000007cffa60: 00000002015f0000 0000000000020000 > 0x0000000007cffa70: 0000000000000048 0000000000000001 > 0x0000000007cffa80: 00000001ffff0000 0000000000bb8501 > 0x0000000007cffa90: 0000000001dbf378 00000000080ea807 > 0x0000000007cffaa0: 0000000007cffb40 0000000007cffb40 > > Instructions: (pc=0x00000000080e51c3) > 0x00000000080e51b3: 4c 8d 44 24 20 48 8b d6 48 8b 41 10 48 83 c1 10 > 0x00000000080e51c3: ff 90 c0 01 00 00 44 8b 1d 08 f2 44 00 45 85 db > > > Stack: [0x0000000000000000,0x0000000000000000], sp=0x0000000007cff9b0, > free space=127998k > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code) > V [jvm.dll+0xe51c3] > > [error occurred during error reporting (printing native stack), id > 0xc0000005] > > > --------------- P R O C E S S --------------- > > Java Threads: ( => current thread ) > 0x0000000010286c00 JavaThread "Thread-135" daemon [_thread_blocked, > id=4892, stack(0x0000000011690000,0x0000000011790000)] > 0x0000000010285400 JavaThread "http-8084-10" daemon [_thread_blocked, > id=5108, stack(0x0000000012010000,0x0000000012110000)] > 0x0000000010287400 JavaThread "http-8084-9" daemon [_thread_blocked, > id=1772, stack(0x00000000149a0000,0x0000000014aa0000)] > 0x000000001028a400 JavaThread "http-8084-8" daemon [_thread_blocked, > id=1656, stack(0x0000000011f10000,0x0000000012010000)] > 0x0000000001dc2c00 JavaThread "http-8084-7" daemon [_thread_blocked, > id=2056, stack(0x0000000011e10000,0x0000000011f10000)] > 0x0000000010288400 JavaThread "http-8084-6" daemon [_thread_blocked, > id=4792, stack(0x0000000011d10000,0x0000000011e10000)] > 0x0000000010286800 JavaThread "MultiThreadedHttpConnectionManager cleanup" > daemon [_thread_blocked, id=3792, > stack(0x0000000012510000,0x0000000012610000)] > 0x000000000f6e8400 JavaThread "http-8084-5" daemon [_thread_blocked, > id=3540, stack(0x0000000011c10000,0x0000000011d10000)] > 0x000000000f6e7800 JavaThread "http-8084-4" daemon [_thread_blocked, > id=4048, stack(0x0000000011b10000,0x0000000011c10000)] > 0x000000000f6e8000 JavaThread "http-8084-3" daemon [_thread_blocked, > id=1932, stack(0x0000000011590000,0x0000000011690000)] > 0x000000000f6e7000 JavaThread "http-8084-2" daemon [_thread_blocked, > id=996, stack(0x0000000011490000,0x0000000011590000)] > 0x0000000001dc6000 JavaThread "http-8084-1" daemon [_thread_blocked, > id=4924, stack(0x0000000011390000,0x0000000011490000)] > 0x0000000001dc5800 JavaThread "TP-Monitor" daemon [_thread_blocked, > id=2288, stack(0x0000000011210000,0x0000000011310000)] > 0x0000000001dc5400 JavaThread "TP-Processor4" daemon [_thread_in_native, > id=4588, stack(0x0000000011110000,0x0000000011210000)] > 0x0000000001dc4c00 JavaThread "TP-Processor3" daemon [_thread_blocked, > id=652, stack(0x0000000011010000,0x0000000011110000)] > 0x0000000001dc4400 JavaThread "TP-Processor2" daemon [_thread_blocked, > id=1996, stack(0x000000000fea0000,0x000000000ffa0000)] > 0x0000000001dc4000 JavaThread "TP-Processor1" daemon [_thread_blocked, > id=3712, stack(0x000000000fda0000,0x000000000fea0000)] > 0x0000000001dc2400 JavaThread "http-8084-Acceptor-0" daemon > [_thread_in_native, id=1904, stack(0x000000000fca0000,0x000000000fda0000)] > 0x0000000001dc3800 JavaThread > "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon > [_thread_blocked, id=4264, stack(0x000000000fba0000,0x000000000fca0000)] > 0x0000000001dc3000 JavaThread "pool-1-thread-1" [_thread_in_Java, id=1264, > stack(0x000000000faa0000,0x000000000fba0000)] > 0x0000000001dc1400 JavaThread "Low Memory Detector" daemon > [_thread_blocked, id=4704, stack(0x000000000ef70000,0x000000000f070000)] > 0x0000000001db0800 JavaThread "CompilerThread1" daemon [_thread_blocked, > id=4848, stack(0x000000000ee70000,0x000000000ef70000)] > 0x0000000001db0400 JavaThread "CompilerThread0" daemon [_thread_blocked, > id=4464, stack(0x000000000ed70000,0x000000000ee70000)] > 0x0000000001dc1000 JavaThread "Attach Listener" daemon [_thread_blocked, > id=1736, stack(0x000000000ec70000,0x000000000ed70000)] > 0x0000000001dc0800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, > id=3360, stack(0x000000000eb70000,0x000000000ec70000)] > 0x0000000001dafc00 JavaThread "Surrogate Locker Thread (CMS)" daemon > [_thread_blocked, id=2212, stack(0x000000000ea70000,0x000000000eb70000)] > 0x0000000001dc0000 JavaThread "Finalizer" daemon [_thread_blocked, > id=3912, stack(0x000000000e970000,0x000000000ea70000)] > 0x0000000001dbfc00 JavaThread "Reference Handler" daemon [_thread_blocked, > id=2920, stack(0x000000000e870000,0x000000000e970000)] > 0x0000000001dbec00 JavaThread "main" [_thread_in_native, id=3620, > stack(0x0000000001e50000,0x0000000001f50000)] > > Other Threads: > 0x0000000001de3000 VMThread [stack: 0x0000000007f00000,0x0000000008000000] > [id=2772] > 0x0000000001de3800 WatcherThread [stack: > 0x000000000f070000,0x000000000f170000] [id=4460] > > =>0x0000000001de2000 (exited) GCTaskThread [stack: > 0x0000000000000000,0x0000000000000000] [id=956] > > VM state:not at safepoint (normal execution) > > VM Mutex/Monitor currently owned by a thread: None > > Heap > par new generation total 153344K, used 136138K [0x000000007fff0000, > 0x000000008a650000, 0x000000008a650000) > eden space 136320K, 91% used [0x000000007fff0000, 0x0000000087985dc0, > 0x0000000088510000) > from space 17024K, 68% used [0x0000000088510000, 0x000000008906cbc0, > 0x00000000895b0000) > to space 17024K, 0% used [0x00000000895b0000, 0x00000000895b0000, > 0x000000008a650000) > concurrent mark-sweep generation total 6121088K, used 2338646K > [0x000000008a650000, 0x00000001ffff0000, 0x00000001ffff0000) > concurrent-mark-sweep perm gen total 41140K, used 25955K > [0x00000001ffff0000, 0x000000020281d000, 0x00000002053f0000) > > Dynamic libraries: > 0x0000000000400000 - 0x000000000042d000 C:\Program > Files\Java\jre1.6.0_07-4\bin\java.exe > 0x0000000077ec0000 - 0x0000000077ff9000 C:\WINDOWS\system32\ntdll.dll > 0x0000000077d40000 - 0x0000000077eb3000 > C:\WINDOWS\system32\kernel32.dll > 0x000007ff7fee0000 - 0x000007ff7ffe5000 > C:\WINDOWS\system32\ADVAPI32.dll > 0x000007ff7fd30000 - 0x000007ff7fec9000 C:\WINDOWS\system32\RPCRT4.dll > 0x000007ff7e9c0000 - 0x000007ff7e9e2000 > C:\WINDOWS\system32\Secur32.dll > 0x0000000008000000 - 0x00000000085aa000 C:\Program > Files\Java\jre1.6.0_07-4\bin\server\jvm.dll > 0x0000000077c20000 - 0x0000000077d2c000 C:\WINDOWS\system32\USER32.dll > 0x000007ff7fc90000 - 0x000007ff7fd2c000 C:\WINDOWS\system32\GDI32.dll > 0x000007ff7e240000 - 0x000007ff7e290000 C:\WINDOWS\system32\WINMM.dll > 0x000007ff7fc00000 - 0x000007ff7fc86000 C:\WINDOWS\system32\msvcrt.dll > 0x000007ff7d500000 - 0x000007ff7d539000 C:\WINDOWS\system32\IMM32.DLL > 0x000007ff77100000 - 0x000007ff7710b000 C:\WINDOWS\system32\rdpsnd.dll > 0x000007ff7d4d0000 - 0x000007ff7d4ea000 C:\WINDOWS\system32\WINSTA.dll > 0x000007ff77370000 - 0x000007ff77408000 > C:\WINDOWS\system32\NETAPI32.dll > 0x000007ff7e380000 - 0x000007ff7e390000 C:\WINDOWS\system32\PSAPI.DLL > 0x0000000010000000 - 0x000000001000a000 C:\Program > Files\Java\jre1.6.0_07-4\bin\hpi.dll > 0x00000000003b0000 - 0x00000000003be000 C:\Program > Files\Java\jre1.6.0_07-4\bin\verify.dll > 0x00000000003c0000 - 0x00000000003e7000 C:\Program > Files\Java\jre1.6.0_07-4\bin\java.dll > 0x0000000001f50000 - 0x0000000001f62000 C:\Program > Files\Java\jre1.6.0_07-4\bin\zip.dll > 0x000000000f980000 - 0x000000000f997000 C:\Program > Files\Java\jre1.6.0_07-4\bin\net.dll > 0x000007ff77310000 - 0x000007ff77340000 C:\WINDOWS\system32\WS2_32.dll > 0x000007ff77140000 - 0x000007ff7714c000 > C:\WINDOWS\system32\WS2HELP.dll > 0x000007ff771b0000 - 0x000007ff7722d000 > C:\WINDOWS\system32\mswsock.dll > 0x000007ff6d200000 - 0x000007ff6d293000 > C:\WINDOWS\system32\hnetcfg.dll > 0x000007ff77170000 - 0x000007ff7717b000 > C:\WINDOWS\System32\wshtcpip.dll > 0x000007ff7e8f0000 - 0x000007ff7e93e000 C:\WINDOWS\system32\DNSAPI.dll > 0x000007ff7e9f0000 - 0x000007ff7e9fb000 C:\WINDOWS\System32\winrnr.dll > 0x000007ff7e950000 - 0x000007ff7e9b6000 > C:\WINDOWS\system32\WLDAP32.dll > 0x000007ff7ea00000 - 0x000007ff7ea07000 > C:\WINDOWS\system32\rasadhlp.dll > 0x0000000068000000 - 0x000000006804c000 C:\WINDOWS\system32\rsaenh.dll > 0x000007ff7c680000 - 0x000007ff7c78a000 > C:\WINDOWS\system32\USERENV.dll > > VM Arguments: > jvm_args: -Xloggc:E:\Apache\Solr\home4\gc.log > -Dsolr.solr.home=E:\Apache\Solr\home4 -Xmx6144m -Xms6144m > -XX:+AggressiveOpts -Xincgc -XX:+UseParNewGC > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > -Djava.util.logging.config.file=E:\Apache\Tomcat\Tomcat > 6.0-4\conf\logging.properties -Djava.endorsed.dirs=E:\Apache\Tomcat\Tomcat > 6.0-4\endorsed -Dcatalina.base=E:\Apache\Tomcat\Tomcat 6.0-4 > -Dcatalina.home=E:\Apache\Tomcat\Tomcat 6.0-4 > -Djava.io.tmpdir=E:\Apache\Tomcat\Tomcat 6.0-4\temp > java_command: org.apache.catalina.startup.Bootstrap start > Launcher Type: SUN_STANDARD > > Environment Variables: > JRE_HOME=C:\Program Files\Java\jre1.6.0_07-4 > CLASSPATH=;E:\Apache\Tomcat\Tomcat 6.0-4\bin\bootstrap.jar > PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program > Files\Intel\DMIX > USERNAME=wpiaseczny > OS=Windows_NT > PROCESSOR_IDENTIFIER=EM64T Family 6 Model 23 Stepping 6, GenuineIntel > > > > --------------- S Y S T E M --------------- > > OS: Windows Server 2003 family Build 3790 Service Pack 2 > > CPU:total 8 (4 cores per cpu, 1 threads per core) family 6 model 7 stepping > 6, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3 > > Memory: 4k page, physical 67107112k(18792216k free), swap > 69649820k(14739920k free) > > vm_info: Java HotSpot(TM) 64-Bit Server VM (10.0-b23) for windows-amd64 JRE > (1.6.0_07-b06), built on Jun 10 2008 01:06:11 by "java_re" with MS VC++ 8.0 > > time: Thu Mar 19 18:00:49 2009 > elapsed time: 94857 seconds
