Hi Shawn, I.m circling back with some new findings with our 2 NUMA issue. After a few iterations, we do see improvement with the useNUMA flag and other JVM setting changes. Here are the current settings, with Java 11:
-XX:+UseNUMA -XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+UseTLAB -XX:G1MaxNewSizePercent=20 -XX:MaxGCPauseMillis=150 -XX:+DisableExplicitGC -XX:+DoEscapeAnalysis -XX:+ParallelRefProcEnabled -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions Compared to previous Java 8 + CMS on 2 NUMA servers, P99 latency has improved over 20%. Thanks, Wei On Mon, Sep 28, 2020 at 4:02 PM Shawn Heisey <apa...@elyograg.org> wrote: > On 9/28/2020 12:17 PM, Wei wrote: > > Thanks Shawn. Looks like Java 11 is the way to go with -XX:+UseNUMA. Do > you > > see any backward compatibility issue for Solr 8 with Java 11? Can we run > > Solr 8 built with JDK 8 in Java 11 JRE, or need to rebuild solr with Java > > 11 JDK? > > I do not know of any problems running the binary release of Solr 8 > (which is most likely built with the Java 8 JDK) with a newer release > like Java 11 or higher. > > I think Sun was really burned by such problems cropping up in the days > of Java 5 and 6, and their developers have worked really hard to make > sure that never happens again. > > If you're running Java 11, you will need to pick a different garbage > collector if you expect the NUMA flag to function. The most recent > releases of Solr are defaulting to G1GC, which as previously mentioned, > did not gain NUMA optimizations until Java 14. > > It is not clear to me whether the NUMA optimizations will work with any > collector other than Parallel until Java 14. You would need to check > Java documentation carefully or ask someone involved with development of > Java. > > If you do see an improvement using the NUMA flag with Java 11, please > let us know exactly what options Solr was started with. > > Thanks, > Shawn >
