It happened at the same CompileTrask on our land (OpenJDK17): Thanks, Jen-Ya --------------- T H R E A D ---------------
Current thread (0x00007ff2f4a44630): JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=14826, stack(0x00007ff2d0429000,0x00007ff2d052a000)] Current CompileTask: C2:80302242 17311 ! 4 com.github.benmanes.caffeine.cache.BoundedLocalCache::put (731 bytes) Stack: [0x00007ff2d0429000,0x00007ff2d052a000], sp=0x00007ff2d0524d70, free space=1007k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0xad1332] PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0xf2 V [libjvm.so+0xad1d80] PhaseIdealLoop::build_loop_late(VectorSet&, Node_List&, Node_Stack&)+0x180 V [libjvm.so+0xad26de] PhaseIdealLoop::build_and_optimize()+0x78e V [libjvm.so+0x5d06c1] PhaseIdealLoop::optimize(PhaseIterGVN&, LoopOptsMode)+0x171 V [libjvm.so+0x5ce5b5] Compile::Optimize()+0xd85 V [libjvm.so+0x5cff09] Compile::Compile(ciEnv*, ciMethod*, int, bool, bool, bool, bool, bool, DirectiveSet*)+0xe49 V [libjvm.so+0x50f7d7] C2Compiler::compile_method(ciEnv*, ciMethod*, int, bool, DirectiveSet*)+0xf7 V [libjvm.so+0x5d8f59] CompileBroker::invoke_compiler_on_method(CompileTask*)+0xf69 V [libjvm.so+0x5d9c00] CompileBroker::compiler_thread_loop()+0x4f0 V [libjvm.so+0xe607b2] JavaThread::thread_main_inner()+0x182 V [libjvm.so+0xe63f7e] Thread::call_run()+0xde V [libjvm.so+0xc1b361] thread_native_entry(Thread*)+0xe1 On Thu, Oct 13, 2022 at 12:28 PM Wei <[email protected]> wrote: > Looks our case is triggered from caffeine cache, different from > https://bugs.openjdk.org/browse/JDK-8285835 where it happens in facet > module. > > > --------------- T H R E A D --------------- > > > Current thread (0x00007f96b1286f40): JavaThread "C2 CompilerThread0" > daemon [_thread_in_native, id=16637, > stack(0x00007f85f50ab000,0x00007f85f51ac000)] > > > > Current CompileTask: > > C2:148683516 28882 ! 4 > com.github.benmanes.caffeine.cache.BoundedLocalCache::put > (731 bytes) > > > Stack: [0x00007f85f50ab000,0x00007f85f51ac000], sp=0x00007f85f51a6e40, > free > space=1007k > > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code) > > V [libjvm.so+0x9e13a1] PhaseIdealLoop::build_loop_late_post_work(Node*, > bool)+0x111 > > V [libjvm.so+0x9e1910] PhaseIdealLoop::build_loop_late(VectorSet&, > Node_List&, Node_Stack&)+0x180 > > V [libjvm.so+0x9e2321] > PhaseIdealLoop::build_and_optimize(LoopOptsMode)+0x841 > > V [libjvm.so+0x5b90da] PhaseIdealLoop::optimize(PhaseIterGVN&, > LoopOptsMode)+0x17a > > V [libjvm.so+0x5b6b5e] Compile::Optimize()+0x84e > > V [libjvm.so+0x5b883c] Compile::Compile(ciEnv*, ciMethod*, int, bool, > bool, bool, bool, bool, DirectiveSet*)+0x13ec > > V [libjvm.so+0x4f87da] C2Compiler::compile_method(ciEnv*, ciMethod*, int, > bool, DirectiveSet*)+0xba > > V [libjvm.so+0x5c0c8d] > CompileBroker::invoke_compiler_on_method(CompileTask*)+0xf8d > > V [libjvm.so+0x5c15d8] CompileBroker::compiler_thread_loop()+0x588 > > V [libjvm.so+0xde6bb8] JavaThread::thread_main_inner()+0x158 > > V [libjvm.so+0xde6cf9] JavaThread::run()+0x129 > > V [libjvm.so+0xde9bce] Thread::call_run()+0x7e > > V [libjvm.so+0xb337b1] thread_native_entry(Thread*)+0xe1 > > On Thu, Oct 13, 2022 at 12:00 PM Wei <[email protected]> wrote: > > > We also observed the same SIGSEGV crash during our test with Solr 9.0.0. > > It happens very rarely though, seem 2 occurences during one month of > > testing. We use Zulu JDK 17.34. > > > > > > - > > > > A fatal error has been detected by the Java Runtime Environment: > > SIGSEGV (0xb) at pc=0x00007f96b852d3a1, pid=16597, tid=16637 > > # JRE version: OpenJDK Runtime Environment Zulu17.34+20-SA (17.0.3+7) > > (build 17.0.3+7-LTS) > > # Java VM: OpenJDK 64-Bit Server VM Zulu17.34+20-SA (17.0.3+7-LTS, > > mixed mode, sharing, tiered, compressed class ptrs, g1 gc, > linux-amd64) > > # Problematic frame: > > # V [libjvm.so+0x9e13a1] > > PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0x111 > > > > > > Do we know which lucene/solr component would trigger this error? > > > > Regards, > > Wei > > > > On Thu, Oct 13, 2022 at 10:51 AM solr <[email protected]> wrote: > > > >> Hi. > >> > >> We tried to upgrade a number of solar images from 8.11 to 9.0.0 this > week > >> and ran into the same problems with the official docker image as well > >> (approximately 30 running pods in Kubernetes, crashing on average once > an > >> hour). > >> > >> We even tried to build a docker image with another base image > >> (bellsoft/liberica-openjdk-debian:17), with the same result (even though > >> the actual error differed somewhat (optimize instead of > >> build_loop_late_post_work)). > >> > >> For now we ended up with building a docker image of solr 9.0.0 based on > >> openjdk:11-jre and it has been running several hours without crashes. > >> Maybe the official docker image should be reverted to this until the > bug is > >> fixed? > >> > >> > >> For reference: here are 2 crashes (hs_err_pidXX.log and replay_pidXX.log > >> can be provided on request): > >> > >> > >> Temurin-17.0.4.1+1: > >> # > >> # A fatal error has been detected by the Java Runtime Environment: > >> # > >> # SIGSEGV (0xb) at pc=0x00007fd81c7e6153, pid=14, tid=85 > >> # > >> # JRE version: OpenJDK Runtime Environment Temurin-17.0.4.1+1 > >> (17.0.4.1+1) (build 17.0.4.1+1) > >> # Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.4.1+1 (17.0.4.1+1, > mixed > >> mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, > >> linux-amd64) > >> # Problematic frame: > >> # V [libjvm.so+0xacc153] > >> PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0x153 > >> # > >> # Core dump will be written. Default location: /core.%e.14.%t > >> # > >> # An error report file with more information is saved as: > >> # /opt/solr-9.0.0/server/hs_err_pid14.log > >> {# > >> # Compiler replay data is saved as: > >> # /opt/solr-9.0.0/server/replay_pid14.log > >> # > >> # If you would like to submit a bug report, please visit: > >> # https://github.com/adoptium/adoptium-support/issues > >> # > >> /opt/scripts/startSolr.sh: line 54: 14 Aborted > >> /opt/solr/bin/solr -f -p ${solrport} -m ${memory} > >> -Dsolr.jetty.request.header.size=65535 -Dsolr.disable.allowUrls=true > >> -Dsolrindex=${profile} -Dlog4j2.formatMsgNoLookups=true > >> -Dfinnapp=${finnapp} -Dsolr.master.host=${masterhost} > -Denable.slave=true > >> -Denable.replication=true -Dsolr.http1=true > >> -Dsolr.disable.shardsWhitelist=true -Dodin.job.host=solr-cl-job > >> -Dodin.bap.host=solr-cl-bap -Dodin.motor.host=solr-cl-motor > >> -Dodin.estate.host=solr-cl-estate > >> > >> > >> bellsoft/liberica-openjdk-debian:17 > >> # > >> # A fatal error has been detected by the Java Runtime Environment: > >> # > >> # SIGSEGV (0xb) at pc=0x00007f96e2515e7f, pid=13, tid=76 > >> # > >> # JRE version: OpenJDK Runtime Environment (17.0.4.1+1) (build > >> 17.0.4.1+1-LTS) > >> # Java VM: OpenJDK 64-Bit Server VM (17.0.4.1+1-LTS, mixed mode, tiered, > >> compressed oops, compressed class ptrs, g1 gc, linux-amd64) > >> # Problematic frame: > >> # V [libjvm.so+0x515e7f] PhaseIdealLoop::optimize(PhaseIterGVN&, > >> LoopOptsMode)+0x13bf > >> # > >> # Core dump will be written. Default location: /core.%e.13.%t > >> # > >> # An error report file with more information is saved as: > >> # /opt/solr-9.0.0/server/hs_err_pid13.log > >> # > >> # Compiler replay data is saved as: > >> # /opt/solr-9.0.0/server/replay_pid13.log > >> # > >> # If you would like to submit a bug report, please visit: > >> # https://bell-sw.com/support > >> # > >> /opt/scripts/startSolr.sh: line 54: 13 Aborted > >> /opt/solr/bin/solr -f -p ${solrport} -m ${memory} > >> -Dsolr.jetty.request.header.size=65535 -Dsolr.disable.allowUrls=true > >> -Dsolrindex=${profile} -Dlog4j2.formatMsgNoLookups=true > >> -Dfinnapp=${finnapp} -Dsolr.master.host=${masterhost} > -Denable.slave=true > >> -Denable.replication=true -Dsolr.http1=true > >> -Dsolr.disable.shardsWhitelist=true -Dodin.job.host=solr-cl-job > >> -Dodin.bap.host=solr-cl-bap -Dodin.motor.host=solr-cl-motor > >> -Dodin.estate.host=solr-cl-estate > >> > >> > >> > >> > >> > >> > >> Regards, > >> > >> > >> Fredrik > >> > >> > >> -- > >> > >> Fredrik Rødland Cell: +47 99 21 98 17 > >> Maisen Pedersens vei 1 Twitter: @fredrikr > >> NO-1363 Høvik, NORWAY flickr: > >> http://www.flickr.com/fmmr/ > >> http://rodland.no about.me http://about.me/fmr > >> > >> >
